2026-02-24T18:02:19.236 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-24T18:02:19.251 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-24T18:02:19.328 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:03:54-rados-wip-yuri8-testing-2026-02-23-2128-distro-default-trial/68602 branch: wip-yuri8-testing-2026-02-23-2128 description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '68602' kernel: branch: distro kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-24_16:03:54-rados-wip-yuri8-testing-2026-02-23-2128-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-yuri8-testing-2026-02-23-2128 ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 618c318a7a0b287ab5aeff207b7bf1d73ef45044 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 618c318a7a0b287ab5aeff207b7bf1d73ef45044 workunit: branch: wip-yuri8-testing-2026-02-23-2128 sha1: 618c318a7a0b287ab5aeff207b7bf1d73ef45044 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 6922 sha1: 618c318a7a0b287ab5aeff207b7bf1d73ef45044 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri8-testing-2026-02-23-2128 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_618c318a7a0b287ab5aeff207b7bf1d73ef45044/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 618c318a7a0b287ab5aeff207b7bf1d73ef45044 targets: trial172.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFKgWvmGFMoTig/R7JDwE604F+9Xer51s9jLSkujxGaqT7paFrIlyWEiqMk0JNI6/15Sjyx+9IRRN48hl3LdNKg= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - osd teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-24_16:03:54 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699 2026-02-24T18:02:19.329 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_618c318a7a0b287ab5aeff207b7bf1d73ef45044/qa; will attempt to use it 2026-02-24T18:02:19.329 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_618c318a7a0b287ab5aeff207b7bf1d73ef45044/qa/tasks 2026-02-24T18:02:19.329 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-24T18:02:19.331 INFO:teuthology.task.internal:Checking packages... 2026-02-24T18:02:19.331 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '618c318a7a0b287ab5aeff207b7bf1d73ef45044' 2026-02-24T18:02:19.331 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-24T18:02:19.331 INFO:teuthology.packaging:ref: None 2026-02-24T18:02:19.331 INFO:teuthology.packaging:tag: None 2026-02-24T18:02:19.331 INFO:teuthology.packaging:branch: wip-yuri8-testing-2026-02-23-2128 2026-02-24T18:02:19.331 INFO:teuthology.packaging:sha1: 618c318a7a0b287ab5aeff207b7bf1d73ef45044 2026-02-24T18:02:19.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-yuri8-testing-2026-02-23-2128 2026-02-24T18:02:19.479 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5507-g618c318a-1jammy 2026-02-24T18:02:19.481 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-24T18:02:19.493 INFO:teuthology.task.internal:no buildpackages task found 2026-02-24T18:02:19.493 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-24T18:02:19.504 INFO:teuthology.task.internal:Saving configuration 2026-02-24T18:02:19.514 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-24T18:02:19.526 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-24T18:02:19.609 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial172.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:03:54-rados-wip-yuri8-testing-2026-02-23-2128-distro-default-trial/68602', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-24 17:58:45.308837', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAZFh+0RfgrF9HftfPwD+oXgtW/wB9ZUS/3OXio/whn4'} 2026-02-24T18:02:19.609 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-24T18:02:19.621 INFO:teuthology.task.internal:roles: ubuntu@trial172.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-24T18:02:19.621 INFO:teuthology.run_tasks:Running task console_log... 2026-02-24T18:02:19.703 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7092fbf1e340>, signals=[15]) 2026-02-24T18:02:19.703 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-24T18:02:19.718 INFO:teuthology.task.internal:Opening connections... 2026-02-24T18:02:19.718 DEBUG:teuthology.task.internal:connecting to ubuntu@trial172.front.sepia.ceph.com 2026-02-24T18:02:19.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:02:19.790 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-24T18:02:19.802 DEBUG:teuthology.orchestra.run.trial172:> uname -m 2026-02-24T18:02:19.869 INFO:teuthology.orchestra.run.trial172.stdout:x86_64 2026-02-24T18:02:19.869 DEBUG:teuthology.orchestra.run.trial172:> cat /etc/os-release 2026-02-24T18:02:19.915 INFO:teuthology.orchestra.run.trial172.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-24T18:02:19.915 INFO:teuthology.orchestra.run.trial172.stdout:NAME="Ubuntu" 2026-02-24T18:02:19.915 INFO:teuthology.orchestra.run.trial172.stdout:VERSION_ID="22.04" 2026-02-24T18:02:19.915 INFO:teuthology.orchestra.run.trial172.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-24T18:02:19.915 INFO:teuthology.orchestra.run.trial172.stdout:VERSION_CODENAME=jammy 2026-02-24T18:02:19.915 INFO:teuthology.orchestra.run.trial172.stdout:ID=ubuntu 2026-02-24T18:02:19.915 INFO:teuthology.orchestra.run.trial172.stdout:ID_LIKE=debian 2026-02-24T18:02:19.915 INFO:teuthology.orchestra.run.trial172.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-24T18:02:19.915 INFO:teuthology.orchestra.run.trial172.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-24T18:02:19.916 INFO:teuthology.orchestra.run.trial172.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-24T18:02:19.916 INFO:teuthology.orchestra.run.trial172.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-24T18:02:19.916 INFO:teuthology.orchestra.run.trial172.stdout:UBUNTU_CODENAME=jammy 2026-02-24T18:02:19.917 INFO:teuthology.lock.ops:Updating trial172.front.sepia.ceph.com on lock server 2026-02-24T18:02:20.035 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-24T18:02:20.046 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-24T18:02:20.058 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-24T18:02:20.058 DEBUG:teuthology.orchestra.run.trial172:> test '!' -e /home/ubuntu/cephtest 2026-02-24T18:02:20.062 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-24T18:02:20.073 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-24T18:02:20.073 DEBUG:teuthology.orchestra.run.trial172:> test -z $(ls -A /var/lib/ceph) 2026-02-24T18:02:20.108 INFO:teuthology.orchestra.run.trial172.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-24T18:02:20.109 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-24T18:02:20.222 INFO:teuthology.run_tasks:Running task kernel... 2026-02-24T18:02:20.242 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-24T18:02:20.242 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-24T18:02:20.242 DEBUG:teuthology.orchestra.run.trial172:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-24T18:02:20.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:02:20.246 DEBUG:teuthology.orchestra.run.trial172:> uname -r 2026-02-24T18:02:20.290 INFO:teuthology.orchestra.run.trial172.stdout:5.15.0-170-generic 2026-02-24T18:02:20.290 INFO:teuthology.task.kernel:Running kernel on trial172: 5.15.0-170-generic 2026-02-24T18:02:20.291 DEBUG:teuthology.orchestra.run.trial172:> sudo apt-get clean 2026-02-24T18:02:20.349 DEBUG:teuthology.orchestra.run.trial172:> sudo apt-get update 2026-02-24T18:02:20.847 INFO:teuthology.orchestra.run.trial172.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-24T18:02:20.937 INFO:teuthology.orchestra.run.trial172.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-24T18:02:21.276 INFO:teuthology.orchestra.run.trial172.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-24T18:02:21.426 INFO:teuthology.orchestra.run.trial172.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-24T18:02:21.566 INFO:teuthology.orchestra.run.trial172.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3269 kB] 2026-02-24T18:02:21.968 INFO:teuthology.orchestra.run.trial172.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [497 kB] 2026-02-24T18:02:21.981 INFO:teuthology.orchestra.run.trial172.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.2 kB] 2026-02-24T18:02:21.981 INFO:teuthology.orchestra.run.trial172.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-24T18:02:22.016 INFO:teuthology.orchestra.run.trial172.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-24T18:02:22.023 INFO:teuthology.orchestra.run.trial172.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.4 kB] 2026-02-24T18:02:22.024 INFO:teuthology.orchestra.run.trial172.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5379 kB] 2026-02-24T18:02:22.183 INFO:teuthology.orchestra.run.trial172.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1022 kB] 2026-02-24T18:02:22.206 INFO:teuthology.orchestra.run.trial172.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [3005 kB] 2026-02-24T18:02:22.293 INFO:teuthology.orchestra.run.trial172.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [429 kB] 2026-02-24T18:02:22.301 INFO:teuthology.orchestra.run.trial172.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-24T18:02:22.301 INFO:teuthology.orchestra.run.trial172.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1018 kB] 2026-02-24T18:02:22.338 INFO:teuthology.orchestra.run.trial172.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe Translation-en [225 kB] 2026-02-24T18:02:22.345 INFO:teuthology.orchestra.run.trial172.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.7 kB] 2026-02-24T18:02:22.346 INFO:teuthology.orchestra.run.trial172.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5204 kB] 2026-02-24T18:02:22.493 INFO:teuthology.orchestra.run.trial172.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [990 kB] 2026-02-24T18:02:26.020 INFO:teuthology.orchestra.run.trial172.stdout:Fetched 23.1 MB in 3s (7793 kB/s) 2026-02-24T18:02:26.567 INFO:teuthology.orchestra.run.trial172.stdout:Reading package lists... 2026-02-24T18:02:26.581 DEBUG:teuthology.orchestra.run.trial172:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-24T18:02:26.613 INFO:teuthology.orchestra.run.trial172.stdout:Reading package lists... 2026-02-24T18:02:26.729 INFO:teuthology.orchestra.run.trial172.stdout:Building dependency tree... 2026-02-24T18:02:26.729 INFO:teuthology.orchestra.run.trial172.stdout:Reading state information... 2026-02-24T18:02:26.820 INFO:teuthology.orchestra.run.trial172.stdout:The following packages were automatically installed and are no longer required: 2026-02-24T18:02:26.820 INFO:teuthology.orchestra.run.trial172.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-24T18:02:26.820 INFO:teuthology.orchestra.run.trial172.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-24T18:02:26.820 INFO:teuthology.orchestra.run.trial172.stdout: sg3-utils-udev 2026-02-24T18:02:26.820 INFO:teuthology.orchestra.run.trial172.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-24T18:02:26.820 INFO:teuthology.orchestra.run.trial172.stdout:The following additional packages will be installed: 2026-02-24T18:02:26.820 INFO:teuthology.orchestra.run.trial172.stdout: linux-generic linux-headers-5.15.0-171 linux-headers-5.15.0-171-generic 2026-02-24T18:02:26.820 INFO:teuthology.orchestra.run.trial172.stdout: linux-headers-generic linux-image-5.15.0-171-generic 2026-02-24T18:02:26.820 INFO:teuthology.orchestra.run.trial172.stdout: linux-modules-5.15.0-171-generic linux-modules-extra-5.15.0-171-generic 2026-02-24T18:02:26.820 INFO:teuthology.orchestra.run.trial172.stdout:Suggested packages: 2026-02-24T18:02:26.821 INFO:teuthology.orchestra.run.trial172.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2026-02-24T18:02:26.834 INFO:teuthology.orchestra.run.trial172.stdout:The following NEW packages will be installed: 2026-02-24T18:02:26.834 INFO:teuthology.orchestra.run.trial172.stdout: linux-headers-5.15.0-171 linux-headers-5.15.0-171-generic 2026-02-24T18:02:26.834 INFO:teuthology.orchestra.run.trial172.stdout: linux-image-5.15.0-171-generic linux-modules-5.15.0-171-generic 2026-02-24T18:02:26.834 INFO:teuthology.orchestra.run.trial172.stdout: linux-modules-extra-5.15.0-171-generic 2026-02-24T18:02:26.835 INFO:teuthology.orchestra.run.trial172.stdout:The following packages will be upgraded: 2026-02-24T18:02:26.835 INFO:teuthology.orchestra.run.trial172.stdout: linux-generic linux-headers-generic linux-image-generic 2026-02-24T18:02:27.247 INFO:teuthology.orchestra.run.trial172.stdout:3 upgraded, 5 newly installed, 0 to remove and 14 not upgraded. 2026-02-24T18:02:27.248 INFO:teuthology.orchestra.run.trial172.stdout:Need to get 113 MB of archives. 2026-02-24T18:02:27.248 INFO:teuthology.orchestra.run.trial172.stdout:After this operation, 585 MB of additional disk space will be used. 2026-02-24T18:02:27.248 INFO:teuthology.orchestra.run.trial172.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-171-generic amd64 5.15.0-171.181 [22.7 MB] 2026-02-24T18:02:28.585 INFO:teuthology.orchestra.run.trial172.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-171-generic amd64 5.15.0-171.181 [11.6 MB] 2026-02-24T18:02:28.916 INFO:teuthology.orchestra.run.trial172.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-171-generic amd64 5.15.0-171.181 [63.9 MB] 2026-02-24T18:02:30.742 INFO:teuthology.orchestra.run.trial172.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.171.160 [1658 B] 2026-02-24T18:02:30.743 INFO:teuthology.orchestra.run.trial172.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.171.160 [2486 B] 2026-02-24T18:02:30.743 INFO:teuthology.orchestra.run.trial172.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-171 all 5.15.0-171.181 [12.4 MB] 2026-02-24T18:02:31.086 INFO:teuthology.orchestra.run.trial172.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-171-generic amd64 5.15.0-171.181 [2849 kB] 2026-02-24T18:02:31.167 INFO:teuthology.orchestra.run.trial172.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.171.160 [2332 B] 2026-02-24T18:02:31.363 INFO:teuthology.orchestra.run.trial172.stdout:Fetched 113 MB in 4s (26.2 MB/s) 2026-02-24T18:02:31.380 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package linux-modules-5.15.0-171-generic. 2026-02-24T18:02:31.556 INFO:teuthology.orchestra.run.trial172.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167442 files and directories currently installed.) 2026-02-24T18:02:31.559 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../0-linux-modules-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T18:02:31.560 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking linux-modules-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T18:02:31.905 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package linux-image-5.15.0-171-generic. 2026-02-24T18:02:31.914 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../1-linux-image-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T18:02:31.935 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking linux-image-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T18:02:31.976 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-171-generic. 2026-02-24T18:02:31.984 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T18:02:31.984 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking linux-modules-extra-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T18:02:33.085 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../3-linux-generic_5.15.0.171.160_amd64.deb ... 2026-02-24T18:02:33.085 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking linux-generic (5.15.0.171.160) over (5.15.0.170.159) ... 2026-02-24T18:02:33.110 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.171.160_amd64.deb ... 2026-02-24T18:02:33.111 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking linux-image-generic (5.15.0.171.160) over (5.15.0.170.159) ... 2026-02-24T18:02:33.127 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package linux-headers-5.15.0-171. 2026-02-24T18:02:33.136 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../5-linux-headers-5.15.0-171_5.15.0-171.181_all.deb ... 2026-02-24T18:02:33.136 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking linux-headers-5.15.0-171 (5.15.0-171.181) ... 2026-02-24T18:02:34.505 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package linux-headers-5.15.0-171-generic. 2026-02-24T18:02:34.519 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../6-linux-headers-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T18:02:34.519 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking linux-headers-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T18:02:35.108 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.171.160_amd64.deb ... 2026-02-24T18:02:35.108 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking linux-headers-generic (5.15.0.171.160) over (5.15.0.170.159) ... 2026-02-24T18:02:35.135 INFO:teuthology.orchestra.run.trial172.stdout:Setting up linux-headers-5.15.0-171 (5.15.0-171.181) ... 2026-02-24T18:02:35.136 INFO:teuthology.orchestra.run.trial172.stdout:Setting up linux-headers-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T18:02:35.138 INFO:teuthology.orchestra.run.trial172.stdout:Setting up linux-headers-generic (5.15.0.171.160) ... 2026-02-24T18:02:35.139 INFO:teuthology.orchestra.run.trial172.stdout:Setting up linux-image-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T18:02:35.906 INFO:teuthology.orchestra.run.trial172.stdout:I: /boot/vmlinuz.old is now a symlink to vmlinuz-5.15.0-170-generic 2026-02-24T18:02:35.906 INFO:teuthology.orchestra.run.trial172.stdout:I: /boot/initrd.img.old is now a symlink to initrd.img-5.15.0-170-generic 2026-02-24T18:02:35.906 INFO:teuthology.orchestra.run.trial172.stdout:I: /boot/vmlinuz is now a symlink to vmlinuz-5.15.0-171-generic 2026-02-24T18:02:35.906 INFO:teuthology.orchestra.run.trial172.stdout:I: /boot/initrd.img is now a symlink to initrd.img-5.15.0-171-generic 2026-02-24T18:02:35.913 INFO:teuthology.orchestra.run.trial172.stdout:Setting up linux-modules-extra-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T18:02:36.705 INFO:teuthology.orchestra.run.trial172.stdout:Setting up linux-modules-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T18:02:37.497 INFO:teuthology.orchestra.run.trial172.stdout:Setting up linux-image-generic (5.15.0.171.160) ... 2026-02-24T18:02:37.498 INFO:teuthology.orchestra.run.trial172.stdout:Setting up linux-generic (5.15.0.171.160) ... 2026-02-24T18:02:37.499 INFO:teuthology.orchestra.run.trial172.stdout:Processing triggers for linux-image-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T18:02:37.505 INFO:teuthology.orchestra.run.trial172.stdout:/etc/kernel/postinst.d/initramfs-tools: 2026-02-24T18:02:37.505 INFO:teuthology.orchestra.run.trial172.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-171-generic 2026-02-24T18:02:44.564 INFO:teuthology.orchestra.run.trial172.stdout:W: Possible missing firmware /lib/firmware/ast_dp501_fw.bin for module ast 2026-02-24T18:02:51.221 INFO:teuthology.orchestra.run.trial172.stdout:/etc/kernel/postinst.d/zz-update-grub: 2026-02-24T18:02:51.222 INFO:teuthology.orchestra.run.trial172.stdout:Sourcing file `/etc/default/grub' 2026-02-24T18:02:51.241 INFO:teuthology.orchestra.run.trial172.stdout:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-24T18:02:51.242 INFO:teuthology.orchestra.run.trial172.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-24T18:02:51.245 INFO:teuthology.orchestra.run.trial172.stdout:Generating grub configuration file ... 2026-02-24T18:02:51.331 INFO:teuthology.orchestra.run.trial172.stdout:Found linux image: /boot/vmlinuz-5.15.0-171-generic 2026-02-24T18:02:51.337 INFO:teuthology.orchestra.run.trial172.stdout:Found initrd image: /boot/initrd.img-5.15.0-171-generic 2026-02-24T18:02:51.512 INFO:teuthology.orchestra.run.trial172.stdout:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-24T18:02:51.517 INFO:teuthology.orchestra.run.trial172.stdout:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-24T18:02:51.561 INFO:teuthology.orchestra.run.trial172.stdout:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-24T18:02:51.566 INFO:teuthology.orchestra.run.trial172.stdout:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-24T18:02:51.642 INFO:teuthology.orchestra.run.trial172.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-24T18:02:51.642 INFO:teuthology.orchestra.run.trial172.stdout:Systems on them will not be added to the GRUB boot configuration. 2026-02-24T18:02:51.642 INFO:teuthology.orchestra.run.trial172.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-24T18:02:51.644 INFO:teuthology.orchestra.run.trial172.stdout:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-24T18:02:51.652 INFO:teuthology.orchestra.run.trial172.stdout:done 2026-02-24T18:02:52.057 INFO:teuthology.orchestra.run.trial172.stdout: 2026-02-24T18:02:52.057 INFO:teuthology.orchestra.run.trial172.stdout:Pending kernel upgrade! 2026-02-24T18:02:52.057 INFO:teuthology.orchestra.run.trial172.stdout: 2026-02-24T18:02:52.057 INFO:teuthology.orchestra.run.trial172.stdout:Running kernel version: 2026-02-24T18:02:52.057 INFO:teuthology.orchestra.run.trial172.stdout: 5.15.0-170-generic 2026-02-24T18:02:52.057 INFO:teuthology.orchestra.run.trial172.stdout: 2026-02-24T18:02:52.057 INFO:teuthology.orchestra.run.trial172.stdout:Diagnostics: 2026-02-24T18:02:52.058 INFO:teuthology.orchestra.run.trial172.stdout: The currently running kernel version is not the expected kernel version 5.15.0-171-generic. 2026-02-24T18:02:52.058 INFO:teuthology.orchestra.run.trial172.stdout: 2026-02-24T18:02:52.058 INFO:teuthology.orchestra.run.trial172.stdout:Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. [Return] 2026-02-24T18:02:52.058 INFO:teuthology.orchestra.run.trial172.stdout: 2026-02-24T18:02:52.058 INFO:teuthology.orchestra.run.trial172.stdout:The processor microcode seems to be up-to-date. 2026-02-24T18:02:52.058 INFO:teuthology.orchestra.run.trial172.stdout: 2026-02-24T18:02:52.058 INFO:teuthology.orchestra.run.trial172.stdout:No services need to be restarted. 2026-02-24T18:02:52.058 INFO:teuthology.orchestra.run.trial172.stdout: 2026-02-24T18:02:52.058 INFO:teuthology.orchestra.run.trial172.stdout:No containers need to be restarted. 2026-02-24T18:02:52.058 INFO:teuthology.orchestra.run.trial172.stdout: 2026-02-24T18:02:52.058 INFO:teuthology.orchestra.run.trial172.stdout:No user sessions are running outdated binaries. 2026-02-24T18:02:52.058 INFO:teuthology.orchestra.run.trial172.stdout: 2026-02-24T18:02:52.059 INFO:teuthology.orchestra.run.trial172.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-24T18:02:52.887 DEBUG:teuthology.orchestra.run.trial172:> dpkg -s linux-image-generic 2026-02-24T18:02:52.898 INFO:teuthology.orchestra.run.trial172.stdout:Package: linux-image-generic 2026-02-24T18:02:52.898 INFO:teuthology.orchestra.run.trial172.stdout:Status: install ok installed 2026-02-24T18:02:52.898 INFO:teuthology.orchestra.run.trial172.stdout:Priority: optional 2026-02-24T18:02:52.898 INFO:teuthology.orchestra.run.trial172.stdout:Section: kernel 2026-02-24T18:02:52.898 INFO:teuthology.orchestra.run.trial172.stdout:Installed-Size: 22 2026-02-24T18:02:52.899 INFO:teuthology.orchestra.run.trial172.stdout:Maintainer: Ubuntu Kernel Team 2026-02-24T18:02:52.899 INFO:teuthology.orchestra.run.trial172.stdout:Architecture: amd64 2026-02-24T18:02:52.899 INFO:teuthology.orchestra.run.trial172.stdout:Source: linux-meta 2026-02-24T18:02:52.899 INFO:teuthology.orchestra.run.trial172.stdout:Version: 5.15.0.171.160 2026-02-24T18:02:52.899 INFO:teuthology.orchestra.run.trial172.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-171), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-24T18:02:52.899 INFO:teuthology.orchestra.run.trial172.stdout:Depends: linux-image-5.15.0-171-generic, linux-modules-extra-5.15.0-171-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-24T18:02:52.899 INFO:teuthology.orchestra.run.trial172.stdout:Recommends: thermald 2026-02-24T18:02:52.899 INFO:teuthology.orchestra.run.trial172.stdout:Description: Generic Linux kernel image 2026-02-24T18:02:52.899 INFO:teuthology.orchestra.run.trial172.stdout: This package will always depend on the latest generic kernel image 2026-02-24T18:02:52.899 INFO:teuthology.orchestra.run.trial172.stdout: available. 2026-02-24T18:02:52.899 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-170-generic Expected: 5.15.0-171-generic 2026-02-24T18:02:52.899 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-24T18:02:52.900 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-24T18:02:52.900 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-24T18:02:52.900 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial172.front.sepia.ceph.com, path=None, version=distro) 2026-02-24T18:02:52.900 DEBUG:teuthology.orchestra.run.trial172:> sudo apt-get clean 2026-02-24T18:02:52.978 DEBUG:teuthology.orchestra.run.trial172:> sudo apt-get update 2026-02-24T18:02:53.431 INFO:teuthology.orchestra.run.trial172.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-24T18:02:53.501 INFO:teuthology.orchestra.run.trial172.stdout:Hit:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-24T18:02:53.581 INFO:teuthology.orchestra.run.trial172.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-24T18:02:53.655 INFO:teuthology.orchestra.run.trial172.stdout:Hit:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-24T18:02:54.330 INFO:teuthology.orchestra.run.trial172.stdout:Reading package lists... 2026-02-24T18:02:54.344 DEBUG:teuthology.orchestra.run.trial172:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-24T18:02:54.377 INFO:teuthology.orchestra.run.trial172.stdout:Reading package lists... 2026-02-24T18:02:54.492 INFO:teuthology.orchestra.run.trial172.stdout:Building dependency tree... 2026-02-24T18:02:54.493 INFO:teuthology.orchestra.run.trial172.stdout:Reading state information... 2026-02-24T18:02:54.584 INFO:teuthology.orchestra.run.trial172.stdout:linux-image-generic is already the newest version (5.15.0.171.160). 2026-02-24T18:02:54.584 INFO:teuthology.orchestra.run.trial172.stdout:linux-image-generic set to manually installed. 2026-02-24T18:02:54.584 INFO:teuthology.orchestra.run.trial172.stdout:The following packages were automatically installed and are no longer required: 2026-02-24T18:02:54.584 INFO:teuthology.orchestra.run.trial172.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-24T18:02:54.584 INFO:teuthology.orchestra.run.trial172.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 2026-02-24T18:02:54.585 INFO:teuthology.orchestra.run.trial172.stdout: linux-headers-5.15.0-164 linux-headers-5.15.0-164-generic 2026-02-24T18:02:54.585 INFO:teuthology.orchestra.run.trial172.stdout: linux-image-5.15.0-164-generic linux-modules-5.15.0-164-generic 2026-02-24T18:02:54.585 INFO:teuthology.orchestra.run.trial172.stdout: linux-modules-extra-5.15.0-164-generic sg3-utils sg3-utils-udev 2026-02-24T18:02:54.585 INFO:teuthology.orchestra.run.trial172.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-24T18:02:54.603 INFO:teuthology.orchestra.run.trial172.stdout:0 upgraded, 0 newly installed, 0 to remove and 14 not upgraded. 2026-02-24T18:02:54.605 DEBUG:teuthology.orchestra.run.trial172:> dpkg -s linux-image-generic 2026-02-24T18:02:54.655 INFO:teuthology.orchestra.run.trial172.stdout:Package: linux-image-generic 2026-02-24T18:02:54.655 INFO:teuthology.orchestra.run.trial172.stdout:Status: install ok installed 2026-02-24T18:02:54.655 INFO:teuthology.orchestra.run.trial172.stdout:Priority: optional 2026-02-24T18:02:54.655 INFO:teuthology.orchestra.run.trial172.stdout:Section: kernel 2026-02-24T18:02:54.655 INFO:teuthology.orchestra.run.trial172.stdout:Installed-Size: 22 2026-02-24T18:02:54.655 INFO:teuthology.orchestra.run.trial172.stdout:Maintainer: Ubuntu Kernel Team 2026-02-24T18:02:54.655 INFO:teuthology.orchestra.run.trial172.stdout:Architecture: amd64 2026-02-24T18:02:54.655 INFO:teuthology.orchestra.run.trial172.stdout:Source: linux-meta 2026-02-24T18:02:54.655 INFO:teuthology.orchestra.run.trial172.stdout:Version: 5.15.0.171.160 2026-02-24T18:02:54.655 INFO:teuthology.orchestra.run.trial172.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-171), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-24T18:02:54.655 INFO:teuthology.orchestra.run.trial172.stdout:Depends: linux-image-5.15.0-171-generic, linux-modules-extra-5.15.0-171-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-24T18:02:54.655 INFO:teuthology.orchestra.run.trial172.stdout:Recommends: thermald 2026-02-24T18:02:54.655 INFO:teuthology.orchestra.run.trial172.stdout:Description: Generic Linux kernel image 2026-02-24T18:02:54.655 INFO:teuthology.orchestra.run.trial172.stdout: This package will always depend on the latest generic kernel image 2026-02-24T18:02:54.655 INFO:teuthology.orchestra.run.trial172.stdout: available. 2026-02-24T18:02:54.656 DEBUG:teuthology.orchestra.run.trial172:> mktemp 2026-02-24T18:02:54.702 INFO:teuthology.orchestra.run.trial172.stdout:/tmp/tmp.WvAnQja3x6 2026-02-24T18:02:54.702 DEBUG:teuthology.orchestra.run.trial172:> sudo cp /boot/grub/grub.cfg /tmp/tmp.WvAnQja3x6 2026-02-24T18:02:54.752 DEBUG:teuthology.orchestra.run.trial172:> sudo chmod 0666 /tmp/tmp.WvAnQja3x6 2026-02-24T18:02:54.854 DEBUG:teuthology.orchestra.remote:trial172:/tmp/tmp.WvAnQja3x6 is 9KB 2026-02-24T18:02:54.862 DEBUG:teuthology.orchestra.run.trial172:> rm -fr /tmp/tmp.WvAnQja3x6 2026-02-24T18:02:54.866 DEBUG:teuthology.orchestra.run.trial172:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2026-02-24T18:02:54.913 DEBUG:teuthology.orchestra.run.trial172:> set -ex 2026-02-24T18:02:54.914 DEBUG:teuthology.orchestra.run.trial172:> sudo dd of=/etc/grub.d/01_ceph_kernel 2026-02-24T18:02:54.914 DEBUG:teuthology.orchestra.run.trial172:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2026-02-24T18:02:54.968 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-171-generic 2026-02-24T18:02:54.969 DEBUG:teuthology.orchestra.run.trial172:> sudo update-grub 2026-02-24T18:02:55.206 INFO:teuthology.orchestra.run.trial172.stderr:Sourcing file `/etc/default/grub' 2026-02-24T18:02:55.225 INFO:teuthology.orchestra.run.trial172.stderr:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-24T18:02:55.225 INFO:teuthology.orchestra.run.trial172.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-24T18:02:55.227 INFO:teuthology.orchestra.run.trial172.stderr:Generating grub configuration file ... 2026-02-24T18:02:55.311 INFO:teuthology.orchestra.run.trial172.stderr:Found linux image: /boot/vmlinuz-5.15.0-171-generic 2026-02-24T18:02:55.317 INFO:teuthology.orchestra.run.trial172.stderr:Found initrd image: /boot/initrd.img-5.15.0-171-generic 2026-02-24T18:02:55.492 INFO:teuthology.orchestra.run.trial172.stderr:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-24T18:02:55.497 INFO:teuthology.orchestra.run.trial172.stderr:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-24T18:02:55.538 INFO:teuthology.orchestra.run.trial172.stderr:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-24T18:02:55.544 INFO:teuthology.orchestra.run.trial172.stderr:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-24T18:02:55.617 INFO:teuthology.orchestra.run.trial172.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-24T18:02:55.617 INFO:teuthology.orchestra.run.trial172.stderr:Systems on them will not be added to the GRUB boot configuration. 2026-02-24T18:02:55.617 INFO:teuthology.orchestra.run.trial172.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-24T18:02:55.619 INFO:teuthology.orchestra.run.trial172.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-24T18:02:55.627 INFO:teuthology.orchestra.run.trial172.stderr:done 2026-02-24T18:02:55.628 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial172 2026-02-24T18:02:55.628 DEBUG:teuthology.orchestra.run.trial172:> sudo bash -c ' 2026-02-24T18:02:55.629 DEBUG:teuthology.orchestra.run.trial172:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-24T18:02:55.629 DEBUG:teuthology.orchestra.run.trial172:> ' 2026-02-24T18:03:25.630 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-24T18:03:25.631 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial172.front.sepia.ceph.com' 2026-02-24T18:03:25.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:04:25.693 DEBUG:teuthology.orchestra.remote:timed out 2026-02-24T18:04:34.701 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial172.front.sepia.ceph.com' 2026-02-24T18:04:34.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:04:34.827 DEBUG:teuthology.orchestra.run.trial172:> true 2026-02-24T18:04:35.214 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial172.front.sepia.ceph.com' 2026-02-24T18:04:35.214 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-171-generic"... 2026-02-24T18:04:35.214 DEBUG:teuthology.orchestra.run.trial172:> uname -r 2026-02-24T18:04:35.263 INFO:teuthology.orchestra.run.trial172.stdout:5.15.0-171-generic 2026-02-24T18:04:35.264 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-171-generic vs 5.15.0-171-generic 2026-02-24T18:04:35.264 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-24T18:04:35.264 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2026-02-24T18:04:36.264 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-24T18:04:36.265 DEBUG:teuthology.orchestra.run.trial172:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-24T18:04:36.278 INFO:teuthology.orchestra.run.trial172.stdout:ttyS1 2026-02-24T18:04:36.299 DEBUG:teuthology.parallel:result is None 2026-02-24T18:04:36.299 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-24T18:04:36.310 INFO:teuthology.task.internal:Creating test directory... 2026-02-24T18:04:36.310 DEBUG:teuthology.orchestra.run.trial172:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-24T18:04:36.344 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-24T18:04:36.359 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-24T18:04:36.375 INFO:teuthology.task.internal:Creating archive directory... 2026-02-24T18:04:36.376 DEBUG:teuthology.orchestra.run.trial172:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-24T18:04:36.395 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-24T18:04:36.406 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-24T18:04:36.406 DEBUG:teuthology.orchestra.run.trial172:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-24T18:04:36.441 INFO:teuthology.orchestra.run.trial172.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:04:36.448 INFO:teuthology.orchestra.run.trial172.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:04:36.449 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-24T18:04:36.461 INFO:teuthology.task.internal:Configuring sudo... 2026-02-24T18:04:36.462 DEBUG:teuthology.orchestra.run.trial172:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-24T18:04:36.501 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-24T18:04:36.518 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-24T18:04:36.519 DEBUG:teuthology.orchestra.run.trial172:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-24T18:04:36.548 DEBUG:teuthology.orchestra.run.trial172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:04:36.596 DEBUG:teuthology.orchestra.run.trial172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:04:36.640 DEBUG:teuthology.orchestra.run.trial172:> set -ex 2026-02-24T18:04:36.640 DEBUG:teuthology.orchestra.run.trial172:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-24T18:04:36.690 DEBUG:teuthology.orchestra.run.trial172:> sudo service rsyslog restart 2026-02-24T18:04:36.750 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-24T18:04:36.762 INFO:teuthology.task.internal:Starting timer... 2026-02-24T18:04:36.762 INFO:teuthology.run_tasks:Running task pcp... 2026-02-24T18:04:36.784 INFO:teuthology.run_tasks:Running task selinux... 2026-02-24T18:04:36.801 DEBUG:teuthology.task.selinux:Excluding trial172: OS 'ubuntu' does not support SELinux 2026-02-24T18:04:36.801 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-24T18:04:36.801 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-24T18:04:36.802 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-24T18:04:36.802 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-24T18:04:36.817 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-24T18:04:36.828 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-24T18:04:36.829 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial172.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-24T18:06:59.288 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial172.front.sepia.ceph.com')] 2026-02-24T18:06:59.290 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial172.front.sepia.ceph.com' 2026-02-24T18:06:59.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:06:59.406 DEBUG:teuthology.orchestra.run.trial172:> true 2026-02-24T18:06:59.648 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial172.front.sepia.ceph.com' 2026-02-24T18:06:59.649 INFO:teuthology.run_tasks:Running task clock... 2026-02-24T18:06:59.658 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-24T18:06:59.658 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-24T18:06:59.658 DEBUG:teuthology.orchestra.run.trial172:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T18:06:59.707 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-24T18:06:59.707 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: Command line: ntpd -gq 2026-02-24T18:06:59.707 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: ---------------------------------------------------- 2026-02-24T18:06:59.708 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: ntp-4 is maintained by Network Time Foundation, 2026-02-24T18:06:59.708 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-24T18:06:59.708 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: corporation. Support and training for ntp-4 are 2026-02-24T18:06:59.708 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: available at https://www.nwtime.org/support 2026-02-24T18:06:59.708 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: ---------------------------------------------------- 2026-02-24T18:06:59.708 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: proto: precision = 0.020 usec (-25) 2026-02-24T18:06:59.708 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: basedate set to 2022-02-04 2026-02-24T18:06:59.708 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: gps base set to 2022-02-06 (week 2196) 2026-02-24T18:06:59.708 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-24T18:06:59.709 INFO:teuthology.orchestra.run.trial172.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-24T18:06:59.709 INFO:teuthology.orchestra.run.trial172.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-24T18:06:59.710 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: restrict ::: KOD does nothing without LIMITED. 2026-02-24T18:06:59.710 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: Listen and drop on 0 v6wildcard [::]:123 2026-02-24T18:06:59.710 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-24T18:06:59.710 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: Listen normally on 2 lo 127.0.0.1:123 2026-02-24T18:06:59.710 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: Listen normally on 3 enp1s0f0 10.20.193.172:123 2026-02-24T18:06:59.710 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: Listen normally on 4 lo [::1]:123 2026-02-24T18:06:59.710 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:64a0%2]:123 2026-02-24T18:06:59.711 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:06:59 ntpd[8832]: Listening on routing socket on fd #22 for interface updates 2026-02-24T18:07:00.707 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:00 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:00.707 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:00 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:00.775 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:00 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:00.775 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:00 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:00.776 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:00 ntpd[8832]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T18:07:00.776 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:00 ntpd[8832]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T18:07:01.707 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:01.707 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:01.707 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:01.708 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:01.708 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:01.708 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:01.715 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:01.715 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:01.715 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T18:07:01.715 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T18:07:01.747 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:01.747 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:01.747 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T18:07:01.747 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T18:07:01.748 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:01.748 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:01.748 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T18:07:01.748 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:01 ntpd[8832]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T18:07:02.707 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:02 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:02.707 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:02 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:02.775 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:02 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:02.775 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:02 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:02.775 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:02 ntpd[8832]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T18:07:02.775 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:02 ntpd[8832]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T18:07:03.722 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:03.724 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:03.724 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:03.724 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:03.724 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:03.725 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:03.725 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:03.725 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:03.725 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T18:07:03.725 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T18:07:03.746 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:03.747 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:03.747 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T18:07:03.747 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T18:07:03.748 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:03.748 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:03.748 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T18:07:03.748 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:03 ntpd[8832]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T18:07:04.707 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:04 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:04.707 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:04 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:04.775 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:04 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:04.775 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:04 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:04.775 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:04 ntpd[8832]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T18:07:04.776 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:04 ntpd[8832]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T18:07:05.707 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:05.707 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:05.708 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:05.708 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:05.708 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:05.708 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:05.714 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:05.714 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:05.714 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T18:07:05.714 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T18:07:05.746 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:05.746 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:05.746 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T18:07:05.746 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T18:07:05.748 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:05.748 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:05.748 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T18:07:05.748 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:05 ntpd[8832]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T18:07:06.707 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:06 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:06.707 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:06 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:06.775 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:06 ntpd[8832]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T18:07:06.775 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:06 ntpd[8832]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T18:07:06.775 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:06 ntpd[8832]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T18:07:06.775 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:06 ntpd[8832]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T18:07:06.776 INFO:teuthology.orchestra.run.trial172.stdout:24 Feb 18:07:06 ntpd[8832]: ntpd: time set +0.060660 s 2026-02-24T18:07:06.776 INFO:teuthology.orchestra.run.trial172.stdout:ntpd: time set +0.060660s 2026-02-24T18:07:06.777 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:06 ntpd[8832]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-24T18:07:06.777 INFO:teuthology.orchestra.run.trial172.stderr:24 Feb 18:07:06 ntpd[8832]: can't open /var/log/ntpstats/loopstats.20260224: Permission denied 2026-02-24T18:07:07.179 INFO:teuthology.orchestra.run.trial172.stdout: remote refid st t when poll reach delay offset jitter 2026-02-24T18:07:07.179 INFO:teuthology.orchestra.run.trial172.stdout:============================================================================== 2026-02-24T18:07:07.180 INFO:teuthology.orchestra.run.trial172.stdout: 204.2.134.173 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-24T18:07:07.180 INFO:teuthology.orchestra.run.trial172.stdout: mail.mlevin.net .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-24T18:07:07.180 INFO:teuthology.orchestra.run.trial172.stdout: ntp1.wiktel.com .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-24T18:07:07.180 INFO:teuthology.orchestra.run.trial172.stdout: 172-234-37-140. .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-24T18:07:07.182 INFO:teuthology.run_tasks:Running task install... 2026-02-24T18:07:07.200 DEBUG:teuthology.task.install:project ceph 2026-02-24T18:07:07.200 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '618c318a7a0b287ab5aeff207b7bf1d73ef45044'}} 2026-02-24T18:07:07.200 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '618c318a7a0b287ab5aeff207b7bf1d73ef45044'} 2026-02-24T18:07:07.201 INFO:teuthology.task.install:Using flavor: default 2026-02-24T18:07:07.209 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-24T18:07:07.209 INFO:teuthology.task.install:extra packages: [] 2026-02-24T18:07:07.211 DEBUG:teuthology.orchestra.run.trial172:> sudo apt-key list | grep Ceph 2026-02-24T18:07:07.276 INFO:teuthology.orchestra.run.trial172.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-24T18:07:07.304 INFO:teuthology.orchestra.run.trial172.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-24T18:07:07.304 INFO:teuthology.orchestra.run.trial172.stdout:uid [ unknown] Ceph.com (release key) 2026-02-24T18:07:07.306 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-osd, ceph-osd-classic, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-24T18:07:07.307 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=618c318a7a0b287ab5aeff207b7bf1d73ef45044 2026-02-24T18:07:07.483 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default/ 2026-02-24T18:07:07.483 INFO:teuthology.task.install.deb:Package version is 20.3.0-5507-g618c318a-1jammy 2026-02-24T18:07:07.630 DEBUG:teuthology.orchestra.run.trial172:> set -ex 2026-02-24T18:07:07.630 DEBUG:teuthology.orchestra.run.trial172:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-24T18:07:07.640 DEBUG:teuthology.orchestra.run.trial172:> sudo apt-get update 2026-02-24T18:07:07.790 INFO:teuthology.orchestra.run.trial172.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy InRelease 2026-02-24T18:07:07.802 INFO:teuthology.orchestra.run.trial172.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy Release [7689 B] 2026-02-24T18:07:07.813 INFO:teuthology.orchestra.run.trial172.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-24T18:07:07.825 INFO:teuthology.orchestra.run.trial172.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 Packages [19.2 kB] 2026-02-24T18:07:08.071 INFO:teuthology.orchestra.run.trial172.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-24T18:07:08.138 INFO:teuthology.orchestra.run.trial172.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-24T18:07:08.210 INFO:teuthology.orchestra.run.trial172.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-24T18:07:08.283 INFO:teuthology.orchestra.run.trial172.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-24T18:07:08.425 INFO:teuthology.orchestra.run.trial172.stdout:Fetched 26.9 kB in 1s (43.0 kB/s) 2026-02-24T18:07:08.949 INFO:teuthology.orchestra.run.trial172.stdout:Reading package lists... 2026-02-24T18:07:08.962 DEBUG:teuthology.orchestra.run.trial172:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=20.3.0-5507-g618c318a-1jammy cephadm=20.3.0-5507-g618c318a-1jammy ceph-mds=20.3.0-5507-g618c318a-1jammy ceph-mgr=20.3.0-5507-g618c318a-1jammy ceph-osd=20.3.0-5507-g618c318a-1jammy ceph-osd-classic=20.3.0-5507-g618c318a-1jammy ceph-common=20.3.0-5507-g618c318a-1jammy ceph-fuse=20.3.0-5507-g618c318a-1jammy ceph-test=20.3.0-5507-g618c318a-1jammy ceph-volume=20.3.0-5507-g618c318a-1jammy radosgw=20.3.0-5507-g618c318a-1jammy python3-rados=20.3.0-5507-g618c318a-1jammy python3-rgw=20.3.0-5507-g618c318a-1jammy python3-cephfs=20.3.0-5507-g618c318a-1jammy python3-rbd=20.3.0-5507-g618c318a-1jammy libcephfs2=20.3.0-5507-g618c318a-1jammy libcephfs-dev=20.3.0-5507-g618c318a-1jammy librados2=20.3.0-5507-g618c318a-1jammy librbd1=20.3.0-5507-g618c318a-1jammy rbd-fuse=20.3.0-5507-g618c318a-1jammy 2026-02-24T18:07:08.994 INFO:teuthology.orchestra.run.trial172.stdout:Reading package lists... 2026-02-24T18:07:09.103 INFO:teuthology.orchestra.run.trial172.stdout:Building dependency tree... 2026-02-24T18:07:09.103 INFO:teuthology.orchestra.run.trial172.stdout:Reading state information... 2026-02-24T18:07:09.192 INFO:teuthology.orchestra.run.trial172.stdout:The following packages were automatically installed and are no longer required: 2026-02-24T18:07:09.192 INFO:teuthology.orchestra.run.trial172.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-24T18:07:09.192 INFO:teuthology.orchestra.run.trial172.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 2026-02-24T18:07:09.192 INFO:teuthology.orchestra.run.trial172.stdout: linux-headers-5.15.0-164 linux-headers-5.15.0-164-generic 2026-02-24T18:07:09.192 INFO:teuthology.orchestra.run.trial172.stdout: linux-image-5.15.0-164-generic linux-modules-5.15.0-164-generic 2026-02-24T18:07:09.192 INFO:teuthology.orchestra.run.trial172.stdout: linux-modules-extra-5.15.0-164-generic sg3-utils sg3-utils-udev 2026-02-24T18:07:09.192 INFO:teuthology.orchestra.run.trial172.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-24T18:07:09.192 INFO:teuthology.orchestra.run.trial172.stdout:The following additional packages will be installed: 2026-02-24T18:07:09.193 INFO:teuthology.orchestra.run.trial172.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-24T18:07:09.193 INFO:teuthology.orchestra.run.trial172.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon jq libcephfs-daemon 2026-02-24T18:07:09.193 INFO:teuthology.orchestra.run.trial172.stdout: libcephfs-proxy2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-24T18:07:09.193 INFO:teuthology.orchestra.run.trial172.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-24T18:07:09.193 INFO:teuthology.orchestra.run.trial172.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2026-02-24T18:07:09.193 INFO:teuthology.orchestra.run.trial172.stdout: nvme-cli python-asyncssh-doc python3-asyncssh python3-cachetools 2026-02-24T18:07:09.193 INFO:teuthology.orchestra.run.trial172.stdout: python3-ceph-argparse python3-ceph-common python3-cheroot python3-cherrypy3 2026-02-24T18:07:09.193 INFO:teuthology.orchestra.run.trial172.stdout: python3-google-auth python3-iniconfig python3-jaraco.classes 2026-02-24T18:07:09.193 INFO:teuthology.orchestra.run.trial172.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2026-02-24T18:07:09.193 INFO:teuthology.orchestra.run.trial172.stdout: python3-joblib python3-kubernetes python3-natsort python3-pluggy 2026-02-24T18:07:09.193 INFO:teuthology.orchestra.run.trial172.stdout: python3-portend python3-prettytable python3-psutil python3-py 2026-02-24T18:07:09.193 INFO:teuthology.orchestra.run.trial172.stdout: python3-pygments python3-pytest python3-repoze.lru python3-requests-oauthlib 2026-02-24T18:07:09.193 INFO:teuthology.orchestra.run.trial172.stdout: python3-routes python3-rsa python3-simplejson python3-sklearn 2026-02-24T18:07:09.194 INFO:teuthology.orchestra.run.trial172.stdout: python3-sklearn-lib python3-tempora python3-threadpoolctl python3-toml 2026-02-24T18:07:09.194 INFO:teuthology.orchestra.run.trial172.stdout: python3-wcwidth python3-webob python3-websocket python3-zc.lockfile 2026-02-24T18:07:09.194 INFO:teuthology.orchestra.run.trial172.stdout: qttranslations5-l10n socat xmlstarlet 2026-02-24T18:07:09.194 INFO:teuthology.orchestra.run.trial172.stdout:Suggested packages: 2026-02-24T18:07:09.194 INFO:teuthology.orchestra.run.trial172.stdout: python3-influxdb liblua5.3-dev luarocks python-natsort-doc python-psutil-doc 2026-02-24T18:07:09.194 INFO:teuthology.orchestra.run.trial172.stdout: subversion python-pygments-doc ttf-bitstream-vera python3-paste python3-dap 2026-02-24T18:07:09.194 INFO:teuthology.orchestra.run.trial172.stdout: python-sklearn-doc ipython3 python-webob-doc 2026-02-24T18:07:09.194 INFO:teuthology.orchestra.run.trial172.stdout:Recommended packages: 2026-02-24T18:07:09.194 INFO:teuthology.orchestra.run.trial172.stdout: btrfs-tools 2026-02-24T18:07:09.227 INFO:teuthology.orchestra.run.trial172.stdout:The following NEW packages will be installed: 2026-02-24T18:07:09.227 INFO:teuthology.orchestra.run.trial172.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-24T18:07:09.227 INFO:teuthology.orchestra.run.trial172.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-24T18:07:09.227 INFO:teuthology.orchestra.run.trial172.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-osd-classic ceph-test 2026-02-24T18:07:09.227 INFO:teuthology.orchestra.run.trial172.stdout: ceph-volume cephadm jq libcephfs-daemon libcephfs-dev libcephfs-proxy2 2026-02-24T18:07:09.227 INFO:teuthology.orchestra.run.trial172.stdout: libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-24T18:07:09.227 INFO:teuthology.orchestra.run.trial172.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-24T18:07:09.227 INFO:teuthology.orchestra.run.trial172.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2026-02-24T18:07:09.228 INFO:teuthology.orchestra.run.trial172.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python3-asyncssh 2026-02-24T18:07:09.228 INFO:teuthology.orchestra.run.trial172.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2026-02-24T18:07:09.228 INFO:teuthology.orchestra.run.trial172.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-24T18:07:09.228 INFO:teuthology.orchestra.run.trial172.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-24T18:07:09.228 INFO:teuthology.orchestra.run.trial172.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-natsort 2026-02-24T18:07:09.228 INFO:teuthology.orchestra.run.trial172.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2026-02-24T18:07:09.228 INFO:teuthology.orchestra.run.trial172.stdout: python3-pygments python3-pytest python3-rados python3-rbd python3-repoze.lru 2026-02-24T18:07:09.228 INFO:teuthology.orchestra.run.trial172.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2026-02-24T18:07:09.228 INFO:teuthology.orchestra.run.trial172.stdout: python3-simplejson python3-sklearn python3-sklearn-lib python3-tempora 2026-02-24T18:07:09.228 INFO:teuthology.orchestra.run.trial172.stdout: python3-threadpoolctl python3-toml python3-wcwidth python3-webob 2026-02-24T18:07:09.228 INFO:teuthology.orchestra.run.trial172.stdout: python3-websocket python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2026-02-24T18:07:09.228 INFO:teuthology.orchestra.run.trial172.stdout: socat xmlstarlet 2026-02-24T18:07:09.307 INFO:teuthology.orchestra.run.trial172.stdout:0 upgraded, 87 newly installed, 0 to remove and 14 not upgraded. 2026-02-24T18:07:09.307 INFO:teuthology.orchestra.run.trial172.stdout:Need to get 693 MB of archives. 2026-02-24T18:07:09.307 INFO:teuthology.orchestra.run.trial172.stdout:After this operation, 2583 MB of additional disk space will be used. 2026-02-24T18:07:09.307 INFO:teuthology.orchestra.run.trial172.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 20.3.0-5507-g618c318a-1jammy [3796 kB] 2026-02-24T18:07:09.414 INFO:teuthology.orchestra.run.trial172.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 20.3.0-5507-g618c318a-1jammy [2910 kB] 2026-02-24T18:07:09.453 INFO:teuthology.orchestra.run.trial172.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 20.3.0-5507-g618c318a-1jammy [883 kB] 2026-02-24T18:07:09.467 INFO:teuthology.orchestra.run.trial172.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 20.3.0-5507-g618c318a-1jammy [359 kB] 2026-02-24T18:07:09.473 INFO:teuthology.orchestra.run.trial172.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 20.3.0-5507-g618c318a-1jammy [32.8 kB] 2026-02-24T18:07:09.474 INFO:teuthology.orchestra.run.trial172.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 20.3.0-5507-g618c318a-1jammy [225 kB] 2026-02-24T18:07:09.477 INFO:teuthology.orchestra.run.trial172.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 20.3.0-5507-g618c318a-1jammy [88.9 kB] 2026-02-24T18:07:09.478 INFO:teuthology.orchestra.run.trial172.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 20.3.0-5507-g618c318a-1jammy [338 kB] 2026-02-24T18:07:09.484 INFO:teuthology.orchestra.run.trial172.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 20.3.0-5507-g618c318a-1jammy [9130 kB] 2026-02-24T18:07:09.582 INFO:teuthology.orchestra.run.trial172.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-24T18:07:09.635 INFO:teuthology.orchestra.run.trial172.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 20.3.0-5507-g618c318a-1jammy [112 kB] 2026-02-24T18:07:09.636 INFO:teuthology.orchestra.run.trial172.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 20.3.0-5507-g618c318a-1jammy [261 kB] 2026-02-24T18:07:09.640 INFO:teuthology.orchestra.run.trial172.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 20.3.0-5507-g618c318a-1jammy [31.6 MB] 2026-02-24T18:07:09.919 INFO:teuthology.orchestra.run.trial172.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-24T18:07:09.930 INFO:teuthology.orchestra.run.trial172.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-24T18:07:09.999 INFO:teuthology.orchestra.run.trial172.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-24T18:07:10.163 INFO:teuthology.orchestra.run.trial172.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 20.3.0-5507-g618c318a-1jammy [4984 kB] 2026-02-24T18:07:10.200 INFO:teuthology.orchestra.run.trial172.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-24T18:07:10.210 INFO:teuthology.orchestra.run.trial172.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-24T18:07:10.237 INFO:teuthology.orchestra.run.trial172.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-24T18:07:10.241 INFO:teuthology.orchestra.run.trial172.stdout:Get:21 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 20.3.0-5507-g618c318a-1jammy [295 kB] 2026-02-24T18:07:10.244 INFO:teuthology.orchestra.run.trial172.stdout:Get:22 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 20.3.0-5507-g618c318a-1jammy [127 kB] 2026-02-24T18:07:10.245 INFO:teuthology.orchestra.run.trial172.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-24T18:07:10.246 INFO:teuthology.orchestra.run.trial172.stdout:Get:24 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 20.3.0-5507-g618c318a-1jammy [934 kB] 2026-02-24T18:07:10.247 INFO:teuthology.orchestra.run.trial172.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-24T18:07:10.248 INFO:teuthology.orchestra.run.trial172.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-24T18:07:10.248 INFO:teuthology.orchestra.run.trial172.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-24T18:07:10.262 INFO:teuthology.orchestra.run.trial172.stdout:Get:28 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 20.3.0-5507-g618c318a-1jammy [7550 kB] 2026-02-24T18:07:10.265 INFO:teuthology.orchestra.run.trial172.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-24T18:07:10.266 INFO:teuthology.orchestra.run.trial172.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-24T18:07:10.272 INFO:teuthology.orchestra.run.trial172.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-24T18:07:10.340 INFO:teuthology.orchestra.run.trial172.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-24T18:07:10.340 INFO:teuthology.orchestra.run.trial172.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-24T18:07:10.340 INFO:teuthology.orchestra.run.trial172.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-24T18:07:10.341 INFO:teuthology.orchestra.run.trial172.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-24T18:07:10.341 INFO:teuthology.orchestra.run.trial172.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-24T18:07:10.342 INFO:teuthology.orchestra.run.trial172.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-24T18:07:10.342 INFO:teuthology.orchestra.run.trial172.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-24T18:07:10.344 INFO:teuthology.orchestra.run.trial172.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-24T18:07:10.345 INFO:teuthology.orchestra.run.trial172.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-24T18:07:10.388 INFO:teuthology.orchestra.run.trial172.stdout:Get:41 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-classic amd64 20.3.0-5507-g618c318a-1jammy [468 MB] 2026-02-24T18:07:10.418 INFO:teuthology.orchestra.run.trial172.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-24T18:07:10.420 INFO:teuthology.orchestra.run.trial172.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-24T18:07:10.420 INFO:teuthology.orchestra.run.trial172.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-24T18:07:10.421 INFO:teuthology.orchestra.run.trial172.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-24T18:07:10.445 INFO:teuthology.orchestra.run.trial172.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-24T18:07:10.447 INFO:teuthology.orchestra.run.trial172.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-24T18:07:10.448 INFO:teuthology.orchestra.run.trial172.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-24T18:07:10.505 INFO:teuthology.orchestra.run.trial172.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-24T18:07:10.505 INFO:teuthology.orchestra.run.trial172.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-24T18:07:10.506 INFO:teuthology.orchestra.run.trial172.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-24T18:07:10.561 INFO:teuthology.orchestra.run.trial172.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-24T18:07:10.561 INFO:teuthology.orchestra.run.trial172.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-24T18:07:10.563 INFO:teuthology.orchestra.run.trial172.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-24T18:07:10.566 INFO:teuthology.orchestra.run.trial172.stdout:Get:55 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-24T18:07:10.570 INFO:teuthology.orchestra.run.trial172.stdout:Get:56 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-24T18:07:10.571 INFO:teuthology.orchestra.run.trial172.stdout:Get:57 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-24T18:07:10.634 INFO:teuthology.orchestra.run.trial172.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-24T18:07:10.636 INFO:teuthology.orchestra.run.trial172.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-24T18:07:10.638 INFO:teuthology.orchestra.run.trial172.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-24T18:07:10.643 INFO:teuthology.orchestra.run.trial172.stdout:Get:61 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-24T18:07:10.706 INFO:teuthology.orchestra.run.trial172.stdout:Get:62 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-24T18:07:10.706 INFO:teuthology.orchestra.run.trial172.stdout:Get:63 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-24T18:07:10.706 INFO:teuthology.orchestra.run.trial172.stdout:Get:64 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-24T18:07:10.708 INFO:teuthology.orchestra.run.trial172.stdout:Get:65 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-24T18:07:10.708 INFO:teuthology.orchestra.run.trial172.stdout:Get:66 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-24T18:07:10.716 INFO:teuthology.orchestra.run.trial172.stdout:Get:67 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-24T18:07:10.779 INFO:teuthology.orchestra.run.trial172.stdout:Get:68 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-24T18:07:10.781 INFO:teuthology.orchestra.run.trial172.stdout:Get:69 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-24T18:07:10.782 INFO:teuthology.orchestra.run.trial172.stdout:Get:70 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-24T18:07:10.783 INFO:teuthology.orchestra.run.trial172.stdout:Get:71 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-24T18:07:20.903 INFO:teuthology.orchestra.run.trial172.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 20.3.0-5507-g618c318a-1jammy [180 kB] 2026-02-24T18:07:20.904 INFO:teuthology.orchestra.run.trial172.stdout:Get:73 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 20.3.0-5507-g618c318a-1jammy [14.1 kB] 2026-02-24T18:07:20.904 INFO:teuthology.orchestra.run.trial172.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 20.3.0-5507-g618c318a-1jammy [1030 kB] 2026-02-24T18:07:20.910 INFO:teuthology.orchestra.run.trial172.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 20.3.0-5507-g618c318a-1jammy [2437 kB] 2026-02-24T18:07:20.922 INFO:teuthology.orchestra.run.trial172.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 20.3.0-5507-g618c318a-1jammy [1072 kB] 2026-02-24T18:07:20.933 INFO:teuthology.orchestra.run.trial172.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 20.3.0-5507-g618c318a-1jammy [190 kB] 2026-02-24T18:07:20.941 INFO:teuthology.orchestra.run.trial172.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 20.3.0-5507-g618c318a-1jammy [49.0 MB] 2026-02-24T18:07:22.246 INFO:teuthology.orchestra.run.trial172.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 20.3.0-5507-g618c318a-1jammy [8626 kB] 2026-02-24T18:07:22.485 INFO:teuthology.orchestra.run.trial172.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 20.3.0-5507-g618c318a-1jammy [14.2 kB] 2026-02-24T18:07:22.486 INFO:teuthology.orchestra.run.trial172.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 20.3.0-5507-g618c318a-1jammy [65.5 MB] 2026-02-24T18:07:24.316 INFO:teuthology.orchestra.run.trial172.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 20.3.0-5507-g618c318a-1jammy [136 kB] 2026-02-24T18:07:24.317 INFO:teuthology.orchestra.run.trial172.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-daemon amd64 20.3.0-5507-g618c318a-1jammy [47.3 kB] 2026-02-24T18:07:24.317 INFO:teuthology.orchestra.run.trial172.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-proxy2 amd64 20.3.0-5507-g618c318a-1jammy [33.1 kB] 2026-02-24T18:07:24.317 INFO:teuthology.orchestra.run.trial172.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 20.3.0-5507-g618c318a-1jammy [44.2 kB] 2026-02-24T18:07:24.318 INFO:teuthology.orchestra.run.trial172.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 20.3.0-5507-g618c318a-1jammy [18.4 MB] 2026-02-24T18:07:24.795 INFO:teuthology.orchestra.run.trial172.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 20.3.0-5507-g618c318a-1jammy [93.7 kB] 2026-02-24T18:07:25.079 INFO:teuthology.orchestra.run.trial172.stdout:Fetched 693 MB in 16s (44.5 MB/s) 2026-02-24T18:07:25.095 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-24T18:07:25.262 INFO:teuthology.orchestra.run.trial172.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 202926 files and directories currently installed.) 2026-02-24T18:07:25.265 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../00-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-24T18:07:25.266 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-24T18:07:25.286 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-24T18:07:25.301 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../01-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-24T18:07:25.301 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-24T18:07:25.318 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-24T18:07:25.329 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../02-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-24T18:07:25.329 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-24T18:07:25.347 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-24T18:07:25.357 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../03-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-24T18:07:25.362 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T18:07:25.393 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-24T18:07:25.403 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../04-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-24T18:07:25.403 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T18:07:25.420 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-24T18:07:25.430 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../05-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-24T18:07:25.431 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T18:07:25.451 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-24T18:07:25.461 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../06-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-24T18:07:25.461 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-24T18:07:25.479 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package librados2. 2026-02-24T18:07:25.489 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../07-librados2_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:25.490 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking librados2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:25.533 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libnbd0. 2026-02-24T18:07:25.543 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../08-libnbd0_1.10.5-1_amd64.deb ... 2026-02-24T18:07:25.543 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-24T18:07:25.558 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package librbd1. 2026-02-24T18:07:25.568 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../09-librbd1_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:25.569 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking librbd1 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:25.607 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libcephfs2. 2026-02-24T18:07:25.617 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../10-libcephfs2_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:25.618 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libcephfs2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:25.638 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-rados. 2026-02-24T18:07:25.648 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../11-python3-rados_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:25.649 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-rados (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:25.666 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-24T18:07:25.676 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../12-python3-ceph-argparse_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T18:07:25.677 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-ceph-argparse (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:25.690 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-cephfs. 2026-02-24T18:07:25.700 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../13-python3-cephfs_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:25.701 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-cephfs (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:25.717 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-24T18:07:25.726 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../14-python3-ceph-common_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T18:07:25.727 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-ceph-common (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:25.745 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-24T18:07:25.755 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../15-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-24T18:07:25.756 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-24T18:07:25.771 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-prettytable. 2026-02-24T18:07:25.781 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../16-python3-prettytable_2.5.0-2_all.deb ... 2026-02-24T18:07:25.781 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-24T18:07:25.796 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-rbd. 2026-02-24T18:07:25.806 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../17-python3-rbd_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:25.806 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-rbd (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:25.824 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-24T18:07:25.834 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../18-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-24T18:07:25.834 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-24T18:07:25.852 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package librgw2. 2026-02-24T18:07:25.862 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../19-librgw2_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:25.863 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking librgw2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:25.963 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-rgw. 2026-02-24T18:07:25.974 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../20-python3-rgw_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:25.974 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-rgw (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:25.990 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-24T18:07:26.000 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../21-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-24T18:07:26.000 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-24T18:07:26.015 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libradosstriper1. 2026-02-24T18:07:26.025 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../22-libradosstriper1_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:26.026 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libradosstriper1 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:26.042 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph-common. 2026-02-24T18:07:26.052 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../23-ceph-common_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:26.052 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph-common (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:26.324 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph-base. 2026-02-24T18:07:26.334 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../24-ceph-base_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:26.339 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph-base (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:26.399 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-24T18:07:26.409 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../25-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-24T18:07:26.410 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-24T18:07:26.424 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-cheroot. 2026-02-24T18:07:26.434 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../26-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-24T18:07:26.434 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-24T18:07:26.452 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-24T18:07:26.462 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../27-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-24T18:07:26.462 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-24T18:07:26.476 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-24T18:07:26.486 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../28-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-24T18:07:26.486 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-24T18:07:26.502 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-24T18:07:26.512 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../29-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-24T18:07:26.512 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-24T18:07:26.527 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-tempora. 2026-02-24T18:07:26.536 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../30-python3-tempora_4.1.2-1_all.deb ... 2026-02-24T18:07:26.537 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-24T18:07:26.552 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-portend. 2026-02-24T18:07:26.562 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../31-python3-portend_3.0.0-1_all.deb ... 2026-02-24T18:07:26.562 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-24T18:07:26.577 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-24T18:07:26.587 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../32-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-24T18:07:26.587 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-24T18:07:26.602 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-24T18:07:26.612 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../33-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-24T18:07:26.613 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-24T18:07:26.637 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-natsort. 2026-02-24T18:07:26.647 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../34-python3-natsort_8.0.2-1_all.deb ... 2026-02-24T18:07:26.647 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-24T18:07:26.664 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-24T18:07:26.674 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../35-ceph-mgr-modules-core_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T18:07:26.674 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph-mgr-modules-core (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:26.705 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-24T18:07:26.715 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../36-libsqlite3-mod-ceph_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:26.716 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libsqlite3-mod-ceph (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:26.731 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph-mgr. 2026-02-24T18:07:26.741 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../37-ceph-mgr_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:26.741 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph-mgr (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:26.763 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph-mon. 2026-02-24T18:07:26.772 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../38-ceph-mon_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:26.773 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph-mon (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:26.838 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-24T18:07:26.849 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../39-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-24T18:07:26.849 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-24T18:07:26.864 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph-osd-classic. 2026-02-24T18:07:26.874 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../40-ceph-osd-classic_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:26.874 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph-osd-classic (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:30.843 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph-osd. 2026-02-24T18:07:30.853 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../41-ceph-osd_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:30.854 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph-osd (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:30.874 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph. 2026-02-24T18:07:30.889 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../42-ceph_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:30.889 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:30.906 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph-fuse. 2026-02-24T18:07:30.921 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../43-ceph-fuse_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:30.922 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph-fuse (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:30.945 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph-mds. 2026-02-24T18:07:30.955 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../44-ceph-mds_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:30.956 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph-mds (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:30.989 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package cephadm. 2026-02-24T18:07:30.999 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../45-cephadm_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:30.999 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking cephadm (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:31.016 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-24T18:07:31.026 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../46-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-24T18:07:31.027 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-24T18:07:31.049 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-24T18:07:31.058 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../47-ceph-mgr-cephadm_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T18:07:31.059 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph-mgr-cephadm (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:31.081 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-24T18:07:31.091 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../48-python3-repoze.lru_0.7-2_all.deb ... 2026-02-24T18:07:31.091 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-24T18:07:31.107 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-routes. 2026-02-24T18:07:31.116 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../49-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-24T18:07:31.117 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-24T18:07:31.138 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-24T18:07:31.148 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../50-ceph-mgr-dashboard_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T18:07:31.148 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph-mgr-dashboard (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:31.696 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-24T18:07:31.707 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../51-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-24T18:07:31.708 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-24T18:07:31.750 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-joblib. 2026-02-24T18:07:31.761 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../52-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-24T18:07:31.761 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-24T18:07:31.789 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-24T18:07:31.799 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../53-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-24T18:07:31.799 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-24T18:07:31.815 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-sklearn. 2026-02-24T18:07:31.825 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../54-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-24T18:07:31.826 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-24T18:07:31.906 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-24T18:07:31.916 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../55-ceph-mgr-diskprediction-local_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T18:07:31.917 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph-mgr-diskprediction-local (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:32.077 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-cachetools. 2026-02-24T18:07:32.088 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../56-python3-cachetools_5.0.0-1_all.deb ... 2026-02-24T18:07:32.088 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-24T18:07:32.104 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-rsa. 2026-02-24T18:07:32.114 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../57-python3-rsa_4.8-1_all.deb ... 2026-02-24T18:07:32.114 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-24T18:07:32.133 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-google-auth. 2026-02-24T18:07:32.143 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../58-python3-google-auth_1.5.1-3_all.deb ... 2026-02-24T18:07:32.143 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-24T18:07:32.161 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-24T18:07:32.171 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../59-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-24T18:07:32.171 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-24T18:07:32.187 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-websocket. 2026-02-24T18:07:32.197 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../60-python3-websocket_1.2.3-1_all.deb ... 2026-02-24T18:07:32.197 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-24T18:07:32.215 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-24T18:07:32.226 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../61-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-24T18:07:32.226 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-24T18:07:32.324 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-24T18:07:32.335 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../62-ceph-mgr-k8sevents_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T18:07:32.335 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph-mgr-k8sevents (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:32.350 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-24T18:07:32.360 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../63-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-24T18:07:32.360 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-24T18:07:32.376 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-24T18:07:32.387 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../64-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-24T18:07:32.387 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-24T18:07:32.403 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package jq. 2026-02-24T18:07:32.413 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../65-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-24T18:07:32.413 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-24T18:07:32.428 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package socat. 2026-02-24T18:07:32.439 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../66-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-24T18:07:32.439 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-24T18:07:32.459 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package xmlstarlet. 2026-02-24T18:07:32.469 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../67-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-24T18:07:32.469 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-24T18:07:32.507 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph-test. 2026-02-24T18:07:32.518 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../68-ceph-test_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:32.518 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph-test (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:33.145 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package ceph-volume. 2026-02-24T18:07:33.156 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../69-ceph-volume_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T18:07:33.156 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking ceph-volume (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:33.180 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libcephfs-daemon. 2026-02-24T18:07:33.190 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../70-libcephfs-daemon_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:33.190 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libcephfs-daemon (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:33.205 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libcephfs-proxy2. 2026-02-24T18:07:33.216 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../71-libcephfs-proxy2_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:33.216 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libcephfs-proxy2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:33.232 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-24T18:07:33.242 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../72-libcephfs-dev_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:33.242 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking libcephfs-dev (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:33.259 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package nvme-cli. 2026-02-24T18:07:33.270 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../73-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-24T18:07:33.270 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-24T18:07:33.303 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-24T18:07:33.313 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../74-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-24T18:07:33.313 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-24T18:07:33.343 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-24T18:07:33.354 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../75-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-24T18:07:33.355 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-24T18:07:33.370 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-pluggy. 2026-02-24T18:07:33.380 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../76-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-24T18:07:33.380 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-24T18:07:33.397 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-psutil. 2026-02-24T18:07:33.408 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../77-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-24T18:07:33.408 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-24T18:07:33.427 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-py. 2026-02-24T18:07:33.437 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../78-python3-py_1.10.0-1_all.deb ... 2026-02-24T18:07:33.438 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-24T18:07:33.459 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-pygments. 2026-02-24T18:07:33.469 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../79-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-24T18:07:33.469 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-24T18:07:33.511 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-toml. 2026-02-24T18:07:33.521 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../80-python3-toml_0.10.2-1_all.deb ... 2026-02-24T18:07:33.521 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-24T18:07:33.536 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-pytest. 2026-02-24T18:07:33.547 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../81-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-24T18:07:33.547 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-24T18:07:33.570 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-simplejson. 2026-02-24T18:07:33.580 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../82-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-24T18:07:33.580 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-24T18:07:33.606 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package python3-webob. 2026-02-24T18:07:33.616 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../83-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-24T18:07:33.616 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-24T18:07:33.633 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-24T18:07:33.643 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../84-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-24T18:07:33.643 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-24T18:07:33.712 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package radosgw. 2026-02-24T18:07:33.721 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../85-radosgw_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:33.722 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking radosgw (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:33.881 INFO:teuthology.orchestra.run.trial172.stdout:Selecting previously unselected package rbd-fuse. 2026-02-24T18:07:33.891 INFO:teuthology.orchestra.run.trial172.stdout:Preparing to unpack .../86-rbd-fuse_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T18:07:33.892 INFO:teuthology.orchestra.run.trial172.stdout:Unpacking rbd-fuse (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:33.920 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-24T18:07:33.987 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-24T18:07:33.988 INFO:teuthology.orchestra.run.trial172.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-24T18:07:34.057 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-24T18:07:34.341 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-24T18:07:34.705 INFO:teuthology.orchestra.run.trial172.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-24T18:07:34.715 INFO:teuthology.orchestra.run.trial172.stdout:Setting up cephadm (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:34.759 INFO:teuthology.orchestra.run.trial172.stdout:Adding system user cephadm....done 2026-02-24T18:07:34.764 INFO:teuthology.orchestra.run.trial172.stdout:usermod: no changes 2026-02-24T18:07:34.769 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-24T18:07:34.842 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-24T18:07:34.843 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-24T18:07:34.918 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-24T18:07:34.994 INFO:teuthology.orchestra.run.trial172.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-24T18:07:34.995 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-24T18:07:35.090 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-24T18:07:35.215 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-24T18:07:35.289 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-24T18:07:35.364 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-ceph-argparse (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:35.429 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-24T18:07:35.430 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-24T18:07:35.431 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-24T18:07:35.432 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-24T18:07:35.433 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-24T18:07:35.553 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-24T18:07:35.632 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libcephfs-proxy2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:35.633 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-24T18:07:35.712 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-24T18:07:35.802 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-24T18:07:36.053 INFO:teuthology.orchestra.run.trial172.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-24T18:07:36.054 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-24T18:07:36.150 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-24T18:07:36.276 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-24T18:07:36.356 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-24T18:07:36.427 INFO:teuthology.orchestra.run.trial172.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-24T18:07:36.427 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-ceph-common (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:36.519 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-24T18:07:37.024 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T18:07:37.029 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-24T18:07:37.103 INFO:teuthology.orchestra.run.trial172.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-24T18:07:37.104 INFO:teuthology.orchestra.run.trial172.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-24T18:07:37.104 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-24T18:07:37.177 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-24T18:07:37.250 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T18:07:37.251 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-24T18:07:37.328 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-24T18:07:37.400 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-24T18:07:37.476 INFO:teuthology.orchestra.run.trial172.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-24T18:07:37.477 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-24T18:07:37.562 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-24T18:07:37.562 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-24T18:07:37.639 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-24T18:07:37.723 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-24T18:07:37.796 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-24T18:07:37.797 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-24T18:07:37.933 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-24T18:07:38.006 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T18:07:38.007 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-24T18:07:38.088 INFO:teuthology.orchestra.run.trial172.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-24T18:07:38.089 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-24T18:07:38.222 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-24T18:07:38.223 INFO:teuthology.orchestra.run.trial172.stdout:Setting up librados2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:38.224 INFO:teuthology.orchestra.run.trial172.stdout:Setting up librgw2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:38.225 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libsqlite3-mod-ceph (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:38.226 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-24T18:07:38.756 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libcephfs2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:38.757 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libradosstriper1 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:38.758 INFO:teuthology.orchestra.run.trial172.stdout:Setting up librbd1 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:38.759 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph-mgr-modules-core (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:38.760 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph-fuse (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:38.838 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-24T18:07:38.838 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-24T18:07:39.224 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libcephfs-dev (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:39.225 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-rados (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:39.226 INFO:teuthology.orchestra.run.trial172.stdout:Setting up libcephfs-daemon (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:39.227 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-rbd (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:39.228 INFO:teuthology.orchestra.run.trial172.stdout:Setting up rbd-fuse (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:39.228 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-rgw (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:39.229 INFO:teuthology.orchestra.run.trial172.stdout:Setting up python3-cephfs (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:39.230 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph-common (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:39.259 INFO:teuthology.orchestra.run.trial172.stdout:Adding group ceph....done 2026-02-24T18:07:39.293 INFO:teuthology.orchestra.run.trial172.stdout:Adding system user ceph....done 2026-02-24T18:07:39.304 INFO:teuthology.orchestra.run.trial172.stdout:Setting system user ceph properties....done 2026-02-24T18:07:39.310 INFO:teuthology.orchestra.run.trial172.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-24T18:07:39.400 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-24T18:07:39.680 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-24T18:07:40.032 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph-test (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:40.032 INFO:teuthology.orchestra.run.trial172.stdout:Setting up radosgw (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:40.276 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-24T18:07:40.277 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-24T18:07:40.612 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph-base (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:40.692 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-24T18:07:41.039 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph-mds (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:41.099 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-24T18:07:41.100 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-24T18:07:41.437 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph-mgr (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:41.514 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-24T18:07:41.514 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-24T18:07:41.875 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph-mgr-k8sevents (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:41.875 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph-osd-classic (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:41.890 INFO:teuthology.orchestra.run.trial172.stdout:update-alternatives: using /usr/bin/ceph-osd-classic to provide /usr/bin/ceph-osd (ceph-osd) in auto mode 2026-02-24T18:07:41.900 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph-mgr-diskprediction-local (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:41.910 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph-mon (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:41.980 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-24T18:07:41.980 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-24T18:07:42.336 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph-mgr-cephadm (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:42.348 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph-mgr-dashboard (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:42.358 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph-osd (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:42.443 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-24T18:07:42.444 INFO:teuthology.orchestra.run.trial172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-24T18:07:42.801 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:42.801 INFO:teuthology.orchestra.run.trial172.stdout:Setting up ceph-volume (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T18:07:42.912 INFO:teuthology.orchestra.run.trial172.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-24T18:07:42.932 INFO:teuthology.orchestra.run.trial172.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.13) ... 2026-02-24T18:07:43.360 INFO:teuthology.orchestra.run.trial172.stdout: 2026-02-24T18:07:43.360 INFO:teuthology.orchestra.run.trial172.stdout:Running kernel seems to be up-to-date. 2026-02-24T18:07:43.360 INFO:teuthology.orchestra.run.trial172.stdout: 2026-02-24T18:07:43.360 INFO:teuthology.orchestra.run.trial172.stdout:The processor microcode seems to be up-to-date. 2026-02-24T18:07:43.360 INFO:teuthology.orchestra.run.trial172.stdout: 2026-02-24T18:07:43.360 INFO:teuthology.orchestra.run.trial172.stdout:No services need to be restarted. 2026-02-24T18:07:43.360 INFO:teuthology.orchestra.run.trial172.stdout: 2026-02-24T18:07:43.360 INFO:teuthology.orchestra.run.trial172.stdout:No containers need to be restarted. 2026-02-24T18:07:43.360 INFO:teuthology.orchestra.run.trial172.stdout: 2026-02-24T18:07:43.360 INFO:teuthology.orchestra.run.trial172.stdout:No user sessions are running outdated binaries. 2026-02-24T18:07:43.360 INFO:teuthology.orchestra.run.trial172.stdout: 2026-02-24T18:07:43.360 INFO:teuthology.orchestra.run.trial172.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-24T18:07:44.193 INFO:teuthology.orchestra.run.trial172.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-24T18:07:44.198 DEBUG:teuthology.parallel:result is None 2026-02-24T18:07:44.199 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=618c318a7a0b287ab5aeff207b7bf1d73ef45044 2026-02-24T18:07:44.368 DEBUG:teuthology.orchestra.run.trial172:> dpkg-query -W -f '${Version}' ceph 2026-02-24T18:07:44.378 INFO:teuthology.orchestra.run.trial172.stdout:20.3.0-5507-g618c318a-1jammy 2026-02-24T18:07:44.379 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5507-g618c318a-1jammy 2026-02-24T18:07:44.379 INFO:teuthology.task.install:The correct ceph version 20.3.0-5507-g618c318a-1jammy is installed. 2026-02-24T18:07:44.381 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-24T18:07:44.381 DEBUG:teuthology.orchestra.run.trial172:> set -ex 2026-02-24T18:07:44.381 DEBUG:teuthology.orchestra.run.trial172:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-24T18:07:44.429 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-24T18:07:44.429 DEBUG:teuthology.orchestra.run.trial172:> set -ex 2026-02-24T18:07:44.429 DEBUG:teuthology.orchestra.run.trial172:> sudo dd of=/usr/bin/daemon-helper 2026-02-24T18:07:44.479 DEBUG:teuthology.orchestra.run.trial172:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-24T18:07:44.529 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-24T18:07:44.529 DEBUG:teuthology.orchestra.run.trial172:> set -ex 2026-02-24T18:07:44.530 DEBUG:teuthology.orchestra.run.trial172:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-24T18:07:44.578 DEBUG:teuthology.orchestra.run.trial172:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-24T18:07:44.629 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-24T18:07:44.629 DEBUG:teuthology.orchestra.run.trial172:> set -ex 2026-02-24T18:07:44.629 DEBUG:teuthology.orchestra.run.trial172:> sudo dd of=/usr/bin/stdin-killer 2026-02-24T18:07:44.681 DEBUG:teuthology.orchestra.run.trial172:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-24T18:07:44.733 INFO:teuthology.run_tasks:Running task workunit... 2026-02-24T18:07:44.763 INFO:tasks.workunit:Pulling workunits from ref 618c318a7a0b287ab5aeff207b7bf1d73ef45044 2026-02-24T18:07:44.764 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-24T18:07:44.764 INFO:tasks.workunit:timeout=3h 2026-02-24T18:07:44.764 INFO:tasks.workunit:cleanup=True 2026-02-24T18:07:44.764 DEBUG:teuthology.orchestra.run.trial172:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-24T18:07:44.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:07:44.778 INFO:teuthology.orchestra.run.trial172.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-24T18:07:44.778 DEBUG:teuthology.orchestra.run.trial172:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T18:07:44.825 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-24T18:07:44.825 DEBUG:teuthology.orchestra.run.trial172:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-24T18:07:44.869 DEBUG:teuthology.orchestra.run.trial172:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 618c318a7a0b287ab5aeff207b7bf1d73ef45044 2026-02-24T18:07:44.920 INFO:tasks.workunit.client.0.trial172.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-24T18:08:07.102 INFO:tasks.workunit.client.0.trial172.stderr:Note: switching to '618c318a7a0b287ab5aeff207b7bf1d73ef45044'. 2026-02-24T18:08:07.103 INFO:tasks.workunit.client.0.trial172.stderr: 2026-02-24T18:08:07.103 INFO:tasks.workunit.client.0.trial172.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-24T18:08:07.103 INFO:tasks.workunit.client.0.trial172.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-24T18:08:07.103 INFO:tasks.workunit.client.0.trial172.stderr:state without impacting any branches by switching back to a branch. 2026-02-24T18:08:07.103 INFO:tasks.workunit.client.0.trial172.stderr: 2026-02-24T18:08:07.104 INFO:tasks.workunit.client.0.trial172.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-24T18:08:07.104 INFO:tasks.workunit.client.0.trial172.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-24T18:08:07.104 INFO:tasks.workunit.client.0.trial172.stderr: 2026-02-24T18:08:07.104 INFO:tasks.workunit.client.0.trial172.stderr: git switch -c 2026-02-24T18:08:07.104 INFO:tasks.workunit.client.0.trial172.stderr: 2026-02-24T18:08:07.104 INFO:tasks.workunit.client.0.trial172.stderr:Or undo this operation with: 2026-02-24T18:08:07.105 INFO:tasks.workunit.client.0.trial172.stderr: 2026-02-24T18:08:07.105 INFO:tasks.workunit.client.0.trial172.stderr: git switch - 2026-02-24T18:08:07.105 INFO:tasks.workunit.client.0.trial172.stderr: 2026-02-24T18:08:07.105 INFO:tasks.workunit.client.0.trial172.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-24T18:08:07.105 INFO:tasks.workunit.client.0.trial172.stderr: 2026-02-24T18:08:07.105 INFO:tasks.workunit.client.0.trial172.stderr:HEAD is now at 618c318a7a0 Merge branch 'stretched-cluster' of https://github.com/leonidc/ceph into wip-yuri8-testing-2026-02-23-2128 2026-02-24T18:08:07.112 DEBUG:teuthology.orchestra.run.trial172:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-24T18:08:07.159 DEBUG:teuthology.orchestra.run.trial172:> set -ex 2026-02-24T18:08:07.159 DEBUG:teuthology.orchestra.run.trial172:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-24T18:08:07.206 INFO:tasks.workunit:Running workunits matching osd on client.0... 2026-02-24T18:08:07.207 INFO:tasks.workunit:Running workunit osd/bad-inc-map.sh... 2026-02-24T18:08:07.208 DEBUG:teuthology.orchestra.run.trial172:workunit test osd/bad-inc-map.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=618c318a7a0b287ab5aeff207b7bf1d73ef45044 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh 2026-02-24T18:08:07.260 INFO:tasks.workunit.client.0.trial172.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-24T18:08:07.276 INFO:tasks.workunit.client.0.trial172.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-24T18:08:07.277 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T18:08:07.277 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T18:08:07.277 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: export PYTHONWARNINGS=ignore 2026-02-24T18:08:07.277 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: PYTHONWARNINGS=ignore 2026-02-24T18:08:07.277 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: export CEPH_CONF=/dev/null 2026-02-24T18:08:07.277 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: CEPH_CONF=/dev/null 2026-02-24T18:08:07.277 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: main: unset CEPH_ARGS 2026-02-24T18:08:07.277 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: main: local code 2026-02-24T18:08:07.277 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2428: main: run td/bad-inc-map 2026-02-24T18:08:07.278 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:8: run: local dir=td/bad-inc-map 2026-02-24T18:08:07.278 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:9: run: shift 2026-02-24T18:08:07.278 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:11: run: export CEPH_MON=127.0.0.1:39157 2026-02-24T18:08:07.278 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:11: run: CEPH_MON=127.0.0.1:39157 2026-02-24T18:08:07.278 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:12: run: export CEPH_ARGS 2026-02-24T18:08:07.278 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:13: run: uuidgen 2026-02-24T18:08:07.279 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:13: run: CEPH_ARGS+='--fsid=af907930-40a2-41b2-9101-fabc75d009fd --auth-supported=none ' 2026-02-24T18:08:07.279 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:14: run: CEPH_ARGS+='--mon-host=127.0.0.1:39157 ' 2026-02-24T18:08:07.279 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:15: run: set -e 2026-02-24T18:08:07.279 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: set 2026-02-24T18:08:07.280 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-24T18:08:07.281 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: local funcs=TEST_bad_inc_map 2026-02-24T18:08:07.281 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:18: run: for func in $funcs 2026-02-24T18:08:07.281 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:19: run: setup td/bad-inc-map 2026-02-24T18:08:07.282 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/bad-inc-map 2026-02-24T18:08:07.282 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/bad-inc-map 2026-02-24T18:08:07.282 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/bad-inc-map 2026-02-24T18:08:07.282 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:08:07.282 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/bad-inc-map KILL 2026-02-24T18:08:07.282 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:08:07.282 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:08:07.282 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:08:07.282 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:08:07.282 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:08:07.283 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:08:07.283 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:08:07.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:08:07.284 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:08:07.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:08:07.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:08:07.286 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:08:07.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:08:07.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:08:07.288 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:08:07.288 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:08:07.288 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:08:07.290 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:08:07.290 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/bad-inc-map 2026-02-24T18:08:07.291 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:08:07.292 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:07.292 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11451 2026-02-24T18:08:07.292 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11451 2026-02-24T18:08:07.293 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:08:07.293 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:08:07.293 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/bad-inc-map 2026-02-24T18:08:07.295 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T18:08:07.295 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:07.295 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11451 2026-02-24T18:08:07.295 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11451 2026-02-24T18:08:07.296 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T18:08:07.298 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-24T18:08:07.298 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-24T18:08:07.298 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T18:08:07.298 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/bad-inc-map 1' TERM HUP INT 2026-02-24T18:08:07.298 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:20: run: TEST_bad_inc_map td/bad-inc-map 2026-02-24T18:08:07.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:26: TEST_bad_inc_map: local dir=td/bad-inc-map 2026-02-24T18:08:07.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:28: TEST_bad_inc_map: run_mon td/bad-inc-map a 2026-02-24T18:08:07.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/bad-inc-map 2026-02-24T18:08:07.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T18:08:07.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T18:08:07.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T18:08:07.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/bad-inc-map/a 2026-02-24T18:08:07.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/bad-inc-map/a --run-dir=td/bad-inc-map 2026-02-24T18:08:07.334 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T18:08:07.334 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:08:07.334 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:08:07.335 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:08:07.335 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:07.335 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11451 2026-02-24T18:08:07.335 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11451/$cluster-$name.asok' 2026-02-24T18:08:07.335 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/bad-inc-map/a '--log-file=td/bad-inc-map/$name.log' '--admin-socket=/tmp/ceph-asok.11451/$cluster-$name.asok' --mon-cluster-log-file=td/bad-inc-map/log --run-dir=td/bad-inc-map '--pid-file=td/bad-inc-map/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T18:08:07.365 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T18:08:07.368 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T18:08:07.368 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:08:07.368 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:08:07.368 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T18:08:07.368 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T18:08:07.369 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:08:07.369 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:08:07.369 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:08:07.369 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:08:07.369 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:07.369 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11451 2026-02-24T18:08:07.369 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11451/ceph-mon.a.asok 2026-02-24T18:08:07.370 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:08:07.370 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11451/ceph-mon.a.asok config get fsid 2026-02-24T18:08:07.422 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T18:08:07.422 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:08:07.422 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:08:07.422 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T18:08:07.422 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T18:08:07.424 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:08:07.424 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:08:07.424 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:08:07.424 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:08:07.424 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:07.424 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11451 2026-02-24T18:08:07.424 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11451/ceph-mon.a.asok 2026-02-24T18:08:07.424 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:08:07.424 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11451/ceph-mon.a.asok config get mon_host 2026-02-24T18:08:07.478 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:29: TEST_bad_inc_map: run_mgr td/bad-inc-map x 2026-02-24T18:08:07.478 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/bad-inc-map 2026-02-24T18:08:07.478 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T18:08:07.478 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T18:08:07.478 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T18:08:07.478 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/bad-inc-map/x 2026-02-24T18:08:07.479 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T18:08:07.589 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T18:08:07.590 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:08:07.590 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:08:07.590 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:08:07.590 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:07.590 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11451 2026-02-24T18:08:07.590 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11451/$cluster-$name.asok' 2026-02-24T18:08:07.591 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:08:07.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/bad-inc-map/x '--log-file=td/bad-inc-map/$name.log' '--admin-socket=/tmp/ceph-asok.11451/$cluster-$name.asok' --run-dir=td/bad-inc-map '--pid-file=td/bad-inc-map/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:08:07.615 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:30: TEST_bad_inc_map: run_osd td/bad-inc-map 0 2026-02-24T18:08:07.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/bad-inc-map 2026-02-24T18:08:07.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:08:07.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T18:08:07.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:08:07.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/bad-inc-map/0 2026-02-24T18:08:07.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=af907930-40a2-41b2-9101-fabc75d009fd --auth-supported=none --mon-host=127.0.0.1:39157 ' 2026-02-24T18:08:07.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:08:07.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:08:07.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:08:07.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/0' 2026-02-24T18:08:07.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/0/journal' 2026-02-24T18:08:07.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:08:07.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:08:07.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2026-02-24T18:08:07.617 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:08:07.617 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:08:07.617 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:08:07.617 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:08:07.617 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:07.618 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11451 2026-02-24T18:08:07.618 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11451/$cluster-$name.asok' 2026-02-24T18:08:07.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11451/$cluster-$name.asok' 2026-02-24T18:08:07.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:08:07.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:08:07.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:08:07.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2026-02-24T18:08:07.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2026-02-24T18:08:07.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:08:07.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:08:07.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:08:07.619 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:08:07.619 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:08:07.619 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:08:07.619 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/bad-inc-map/0 2026-02-24T18:08:07.619 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:08:07.620 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dd1ece14-71a5-4f5d-98d5-0b1792ed85fb 2026-02-24T18:08:07.620 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 dd1ece14-71a5-4f5d-98d5-0b1792ed85fb' 2026-02-24T18:08:07.620 INFO:tasks.workunit.client.0.trial172.stdout:add osd0 dd1ece14-71a5-4f5d-98d5-0b1792ed85fb 2026-02-24T18:08:07.620 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:08:07.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAH6Z1pv3ZIJRAAWcVXwmofyX+LtG0p0+l5PQ== 2026-02-24T18:08:07.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAH6Z1pv3ZIJRAAWcVXwmofyX+LtG0p0+l5PQ=="}' 2026-02-24T18:08:07.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dd1ece14-71a5-4f5d-98d5-0b1792ed85fb -i td/bad-inc-map/0/new.json 2026-02-24T18:08:07.742 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:08:07.750 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/bad-inc-map/0/new.json 2026-02-24T18:08:07.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=af907930-40a2-41b2-9101-fabc75d009fd --auth-supported=none --mon-host=127.0.0.1:39157 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/0 --osd-journal=td/bad-inc-map/0/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.11451/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAH6Z1pv3ZIJRAAWcVXwmofyX+LtG0p0+l5PQ== --osd-uuid dd1ece14-71a5-4f5d-98d5-0b1792ed85fb 2026-02-24T18:08:07.767 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:07.761+0000 7fcc5c52ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:07.769 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:07.761+0000 7fcc5c52ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:07.770 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:07.765+0000 7fcc5c52ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:07.771 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:07.765+0000 7fcc5c52ba40 -1 bdev(0x5598e19a0400 td/bad-inc-map/0/block) open stat got: (1) Operation not permitted 2026-02-24T18:08:07.771 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:07.765+0000 7fcc5c52ba40 -1 bluestore(td/bad-inc-map/0) _read_fsid unparsable uuid 2026-02-24T18:08:08.567 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/bad-inc-map/0/keyring 2026-02-24T18:08:08.567 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:08:08.568 INFO:tasks.workunit.client.0.trial172.stdout:adding osd0 key to auth repository 2026-02-24T18:08:08.569 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T18:08:08.569 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/bad-inc-map/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:08:08.691 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:08:08.692 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T18:08:08.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=af907930-40a2-41b2-9101-fabc75d009fd --auth-supported=none --mon-host=127.0.0.1:39157 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/0 --osd-journal=td/bad-inc-map/0/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.11451/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:08:08.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:08:08.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:08:08.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:08:08.713 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:08.705+0000 7f2e2afbea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:08.715 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:08.709+0000 7f2e2afbea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:08.717 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:08.709+0000 7f2e2afbea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:08.801 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T18:08:08.801 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:08:08.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:08:08.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:08:08.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:08:08.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:08.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:08:08.802 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:08:08.803 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:08.803 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:08:08.907 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:08:09.167 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:09.161+0000 7f2e2afbea40 -1 Falling back to public interface 2026-02-24T18:08:09.831 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:09.825+0000 7f2e2afbea40 -1 osd.0 0 log_to_monitors true 2026-02-24T18:08:09.908 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:08:09.908 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:09.909 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:08:09.910 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:08:09.910 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:09.910 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:08:10.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:08:11.021 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:08:11.021 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:11.021 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:08:11.021 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:08:11.021 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:11.021 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:08:11.234 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/601747540,v1:127.0.0.1:6801/601747540] [v2:127.0.0.1:6802/601747540,v1:127.0.0.1:6803/601747540] exists,up dd1ece14-71a5-4f5d-98d5-0b1792ed85fb 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:31: TEST_bad_inc_map: run_osd td/bad-inc-map 1 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/bad-inc-map 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/bad-inc-map/1 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=af907930-40a2-41b2-9101-fabc75d009fd --auth-supported=none --mon-host=127.0.0.1:39157 ' 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/1' 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/1/journal' 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:08:11.235 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11451 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11451/$cluster-$name.asok' 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11451/$cluster-$name.asok' 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:08:11.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/bad-inc-map/1 2026-02-24T18:08:11.237 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:08:11.237 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c34be280-715d-4984-a75a-f834bd2d4a3a 2026-02-24T18:08:11.237 INFO:tasks.workunit.client.0.trial172.stdout:add osd1 c34be280-715d-4984-a75a-f834bd2d4a3a 2026-02-24T18:08:11.238 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 c34be280-715d-4984-a75a-f834bd2d4a3a' 2026-02-24T18:08:11.238 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:08:11.249 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAL6Z1pGGLKDhAA50xsOsF0QvFi8q2LWGXdKA== 2026-02-24T18:08:11.249 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAL6Z1pGGLKDhAA50xsOsF0QvFi8q2LWGXdKA=="}' 2026-02-24T18:08:11.249 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c34be280-715d-4984-a75a-f834bd2d4a3a -i td/bad-inc-map/1/new.json 2026-02-24T18:08:11.467 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:08:11.475 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/bad-inc-map/1/new.json 2026-02-24T18:08:11.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=af907930-40a2-41b2-9101-fabc75d009fd --auth-supported=none --mon-host=127.0.0.1:39157 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/1 --osd-journal=td/bad-inc-map/1/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.11451/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAL6Z1pGGLKDhAA50xsOsF0QvFi8q2LWGXdKA== --osd-uuid c34be280-715d-4984-a75a-f834bd2d4a3a 2026-02-24T18:08:11.491 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:11.485+0000 7fd62c7c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:11.492 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:11.485+0000 7fd62c7c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:11.493 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:11.485+0000 7fd62c7c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:11.494 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:11.489+0000 7fd62c7c6a40 -1 bdev(0x560d1bb40400 td/bad-inc-map/1/block) open stat got: (1) Operation not permitted 2026-02-24T18:08:11.494 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:11.489+0000 7fd62c7c6a40 -1 bluestore(td/bad-inc-map/1) _read_fsid unparsable uuid 2026-02-24T18:08:12.136 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/bad-inc-map/1/keyring 2026-02-24T18:08:12.136 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:08:12.137 INFO:tasks.workunit.client.0.trial172.stdout:adding osd1 key to auth repository 2026-02-24T18:08:12.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T18:08:12.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/bad-inc-map/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:08:12.423 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:08:12.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T18:08:12.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=af907930-40a2-41b2-9101-fabc75d009fd --auth-supported=none --mon-host=127.0.0.1:39157 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/1 --osd-journal=td/bad-inc-map/1/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.11451/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:08:12.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:08:12.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:08:12.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:08:12.445 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:12.437+0000 7fa61b448a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:12.447 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:12.441+0000 7fa61b448a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:12.448 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:12.441+0000 7fa61b448a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:12.653 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T18:08:12.653 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:08:12.653 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:08:12.653 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:08:12.653 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:08:12.653 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:12.654 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:08:12.654 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:08:12.655 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:12.655 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:08:12.867 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:12.861+0000 7fa61b448a40 -1 Falling back to public interface 2026-02-24T18:08:12.868 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:08:13.511 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:13.505+0000 7fa61b448a40 -1 osd.1 0 log_to_monitors true 2026-02-24T18:08:13.870 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:08:13.870 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:13.870 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:08:13.871 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:08:13.871 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:13.871 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:08:14.087 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:08:15.089 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:08:15.089 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:15.090 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:08:15.090 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:08:15.090 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:15.091 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:08:15.295 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3109221026,v1:127.0.0.1:6811/3109221026] [v2:127.0.0.1:6812/3109221026,v1:127.0.0.1:6813/3109221026] exists,up c34be280-715d-4984-a75a-f834bd2d4a3a 2026-02-24T18:08:15.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:08:15.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:08:15.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:08:15.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:32: TEST_bad_inc_map: run_osd td/bad-inc-map 2 2026-02-24T18:08:15.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/bad-inc-map 2026-02-24T18:08:15.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:08:15.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T18:08:15.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:08:15.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/bad-inc-map/2 2026-02-24T18:08:15.300 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=af907930-40a2-41b2-9101-fabc75d009fd --auth-supported=none --mon-host=127.0.0.1:39157 ' 2026-02-24T18:08:15.300 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:08:15.300 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:08:15.300 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:08:15.300 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/2' 2026-02-24T18:08:15.300 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/2/journal' 2026-02-24T18:08:15.300 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:08:15.300 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:08:15.300 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2026-02-24T18:08:15.300 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:08:15.301 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:08:15.301 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:08:15.301 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:08:15.301 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:15.301 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11451 2026-02-24T18:08:15.301 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11451/$cluster-$name.asok' 2026-02-24T18:08:15.301 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11451/$cluster-$name.asok' 2026-02-24T18:08:15.301 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:08:15.301 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:08:15.301 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:08:15.301 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2026-02-24T18:08:15.302 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2026-02-24T18:08:15.302 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:08:15.302 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:08:15.302 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:08:15.302 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:08:15.302 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:08:15.302 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:08:15.302 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/bad-inc-map/2 2026-02-24T18:08:15.302 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:08:15.303 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d937a7e4-0e8c-41d1-bfdb-b8b5c8f55787 2026-02-24T18:08:15.303 INFO:tasks.workunit.client.0.trial172.stdout:add osd2 d937a7e4-0e8c-41d1-bfdb-b8b5c8f55787 2026-02-24T18:08:15.304 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 d937a7e4-0e8c-41d1-bfdb-b8b5c8f55787' 2026-02-24T18:08:15.304 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:08:15.311 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAP6Z1ps2l7EhAA+qUOhIJraiw/tAlE68UC1A== 2026-02-24T18:08:15.311 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAP6Z1ps2l7EhAA+qUOhIJraiw/tAlE68UC1A=="}' 2026-02-24T18:08:15.311 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d937a7e4-0e8c-41d1-bfdb-b8b5c8f55787 -i td/bad-inc-map/2/new.json 2026-02-24T18:08:15.527 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:08:15.536 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/bad-inc-map/2/new.json 2026-02-24T18:08:15.537 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=af907930-40a2-41b2-9101-fabc75d009fd --auth-supported=none --mon-host=127.0.0.1:39157 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/2 --osd-journal=td/bad-inc-map/2/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.11451/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAP6Z1ps2l7EhAA+qUOhIJraiw/tAlE68UC1A== --osd-uuid d937a7e4-0e8c-41d1-bfdb-b8b5c8f55787 2026-02-24T18:08:15.552 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:15.545+0000 7f0f470d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:15.553 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:15.549+0000 7f0f470d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:15.554 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:15.549+0000 7f0f470d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:15.555 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:15.549+0000 7f0f470d7a40 -1 bdev(0x55f7edb70400 td/bad-inc-map/2/block) open stat got: (1) Operation not permitted 2026-02-24T18:08:15.555 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:15.549+0000 7f0f470d7a40 -1 bluestore(td/bad-inc-map/2) _read_fsid unparsable uuid 2026-02-24T18:08:16.188 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/bad-inc-map/2/keyring 2026-02-24T18:08:16.188 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:08:16.189 INFO:tasks.workunit.client.0.trial172.stdout:adding osd2 key to auth repository 2026-02-24T18:08:16.189 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T18:08:16.189 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/bad-inc-map/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:08:16.479 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:08:16.480 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T18:08:16.480 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=af907930-40a2-41b2-9101-fabc75d009fd --auth-supported=none --mon-host=127.0.0.1:39157 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/2 --osd-journal=td/bad-inc-map/2/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.11451/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:08:16.480 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:08:16.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:08:16.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:08:16.501 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:16.493+0000 7f43a2b31a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:16.503 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:16.497+0000 7f43a2b31a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:16.504 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:16.497+0000 7f43a2b31a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:16.692 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T18:08:16.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:08:16.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:08:16.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:08:16.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:08:16.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:16.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:08:16.693 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:08:16.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:16.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:08:16.875 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:16.869+0000 7f43a2b31a40 -1 Falling back to public interface 2026-02-24T18:08:16.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:08:17.474 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:17.469+0000 7f43a2b31a40 -1 osd.2 0 log_to_monitors true 2026-02-24T18:08:17.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:08:17.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:17.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:08:17.900 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:08:17.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:17.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:08:18.117 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2127790447,v1:127.0.0.1:6819/2127790447] [v2:127.0.0.1:6820/2127790447,v1:127.0.0.1:6821/2127790447] exists,up d937a7e4-0e8c-41d1-bfdb-b8b5c8f55787 2026-02-24T18:08:18.118 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:08:18.118 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:08:18.118 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:08:18.118 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:34: TEST_bad_inc_map: ceph config set osd.2 osd_inject_bad_map_crc_probability 1 2026-02-24T18:08:18.334 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:37: TEST_bad_inc_map: create_pool foo 8 2026-02-24T18:08:18.334 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 8 2026-02-24T18:08:18.605 INFO:tasks.workunit.client.0.trial172.stderr:pool 'foo' already exists 2026-02-24T18:08:18.614 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T18:08:19.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:38: TEST_bad_inc_map: ceph osd pool set foo min_size 1 2026-02-24T18:08:19.923 INFO:tasks.workunit.client.0.trial172.stderr:set pool 1 min_size to 1 2026-02-24T18:08:19.937 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:39: TEST_bad_inc_map: ceph osd pool set foo min_size 2 2026-02-24T18:08:20.249 INFO:tasks.workunit.client.0.trial172.stderr:set pool 1 min_size to 2 2026-02-24T18:08:20.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:41: TEST_bad_inc_map: sleep 5 2026-02-24T18:08:25.262 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:44: TEST_bad_inc_map: TIMEOUT=10 2026-02-24T18:08:25.262 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:44: TEST_bad_inc_map: wait_for_osd up 0 2026-02-24T18:08:25.262 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:08:25.262 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:08:25.263 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:08:25.263 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:08:25.263 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 10 )) 2026-02-24T18:08:25.264 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:08:25.265 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:08:25.265 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:25.265 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:08:25.475 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 5 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/601747540,v1:127.0.0.1:6801/601747540] [v2:127.0.0.1:6802/601747540,v1:127.0.0.1:6803/601747540] exists,up dd1ece14-71a5-4f5d-98d5-0b1792ed85fb 2026-02-24T18:08:25.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:08:25.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:08:25.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:08:25.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:45: TEST_bad_inc_map: TIMEOUT=10 2026-02-24T18:08:25.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:45: TEST_bad_inc_map: wait_for_osd up 1 2026-02-24T18:08:25.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:08:25.477 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:08:25.477 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:08:25.477 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:08:25.477 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 10 )) 2026-02-24T18:08:25.477 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:08:25.478 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:08:25.478 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:25.478 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:08:25.689 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 9 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3109221026,v1:127.0.0.1:6811/3109221026] [v2:127.0.0.1:6812/3109221026,v1:127.0.0.1:6813/3109221026] exists,up c34be280-715d-4984-a75a-f834bd2d4a3a 2026-02-24T18:08:25.690 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:08:25.690 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:08:25.690 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:08:25.691 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:46: TEST_bad_inc_map: TIMEOUT=10 2026-02-24T18:08:25.691 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:46: TEST_bad_inc_map: wait_for_osd up 2 2026-02-24T18:08:25.691 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:08:25.691 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:08:25.691 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:08:25.691 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:08:25.691 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 10 )) 2026-02-24T18:08:25.691 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:08:25.692 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:08:25.692 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:25.692 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:08:25.909 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 13 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2127790447,v1:127.0.0.1:6819/2127790447] [v2:127.0.0.1:6820/2127790447,v1:127.0.0.1:6821/2127790447] exists,up d937a7e4-0e8c-41d1-bfdb-b8b5c8f55787 2026-02-24T18:08:25.910 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:08:25.910 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:08:25.910 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:08:25.910 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:49: TEST_bad_inc_map: grep 'injecting map crc failure' td/bad-inc-map/osd.2.log 2026-02-24T18:08:25.911 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:18.645+0000 7f4391ad1640 -1 osd.2 13 handle_osd_map injecting map crc failure 2026-02-24T18:08:25.911 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:18.749+0000 7f4391ad1640 -1 osd.2 15 handle_osd_map injecting map crc failure 2026-02-24T18:08:25.911 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:18.949+0000 7f4391ad1640 -1 osd.2 16 handle_osd_map injecting map crc failure 2026-02-24T18:08:25.911 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:19.817+0000 7f438facd640 -1 osd.2 17 handle_osd_map injecting map crc failure 2026-02-24T18:08:25.911 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:19.921+0000 7f4391ad1640 -1 osd.2 18 handle_osd_map injecting map crc failure 2026-02-24T18:08:25.911 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:19.921+0000 7f4391ad1640 -1 osd.2 18 handle_osd_map injecting map crc failure 2026-02-24T18:08:25.912 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:20.141+0000 7f438facd640 -1 osd.2 19 handle_osd_map injecting map crc failure 2026-02-24T18:08:25.912 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:20.141+0000 7f438facd640 -1 osd.2 19 handle_osd_map injecting map crc failure 2026-02-24T18:08:25.912 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:20.241+0000 7f438facd640 -1 osd.2 20 handle_osd_map injecting map crc failure 2026-02-24T18:08:25.912 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:20.245+0000 7f4391ad1640 -1 osd.2 20 handle_osd_map injecting map crc failure 2026-02-24T18:08:25.912 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:50: TEST_bad_inc_map: grep 'bailing because last' td/bad-inc-map/osd.2.log 2026-02-24T18:08:25.913 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:18.645+0000 7f4391ad1640 10 osd.2 13 handle_osd_map bailing because last < start (13<14) 2026-02-24T18:08:25.913 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:18.749+0000 7f4391ad1640 10 osd.2 15 handle_osd_map bailing because last < start (15<16) 2026-02-24T18:08:25.913 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:18.949+0000 7f4391ad1640 10 osd.2 16 handle_osd_map bailing because last < start (16<17) 2026-02-24T18:08:25.913 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:19.817+0000 7f438facd640 10 osd.2 17 handle_osd_map bailing because last < start (17<18) 2026-02-24T18:08:25.913 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:19.921+0000 7f4391ad1640 10 osd.2 18 handle_osd_map bailing because last < start (18<19) 2026-02-24T18:08:25.913 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:19.921+0000 7f4391ad1640 10 osd.2 18 handle_osd_map bailing because last < start (18<19) 2026-02-24T18:08:25.913 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:20.141+0000 7f438facd640 10 osd.2 19 handle_osd_map bailing because last < start (19<20) 2026-02-24T18:08:25.913 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:20.141+0000 7f438facd640 10 osd.2 19 handle_osd_map bailing because last < start (19<20) 2026-02-24T18:08:25.914 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:20.245+0000 7f438facd640 10 osd.2 20 handle_osd_map bailing because last < start (20<21) 2026-02-24T18:08:25.914 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:08:20.245+0000 7f4391ad1640 10 osd.2 20 handle_osd_map bailing because last < start (20<21) 2026-02-24T18:08:25.914 INFO:tasks.workunit.client.0.trial172.stdout:success 2026-02-24T18:08:25.914 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:52: TEST_bad_inc_map: echo success 2026-02-24T18:08:25.915 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:54: TEST_bad_inc_map: delete_pool foo 2026-02-24T18:08:25.915 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=foo 2026-02-24T18:08:25.915 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete foo foo --yes-i-really-really-mean-it 2026-02-24T18:08:26.177 INFO:tasks.workunit.client.0.trial172.stderr:pool 'foo' does not exist 2026-02-24T18:08:26.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:55: TEST_bad_inc_map: kill_daemons td/bad-inc-map 2026-02-24T18:08:26.187 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:08:26.188 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:08:26.188 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:08:26.188 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:08:26.188 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:08:31.503 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:08:31.503 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:21: run: teardown td/bad-inc-map 2026-02-24T18:08:31.503 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/bad-inc-map 2026-02-24T18:08:31.504 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:08:31.504 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/bad-inc-map KILL 2026-02-24T18:08:31.504 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:08:31.504 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:08:31.505 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:08:31.505 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:08:31.505 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:08:31.510 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:08:31.511 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:08:31.511 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:08:31.512 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:08:31.513 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:08:31.513 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:08:31.513 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:08:31.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:08:31.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:08:31.515 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:08:31.515 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:08:31.516 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:08:31.517 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:08:31.517 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/bad-inc-map 2026-02-24T18:08:31.527 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:08:31.527 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:31.527 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11451 2026-02-24T18:08:31.527 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11451 2026-02-24T18:08:31.529 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:08:31.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:08:31.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2429: main: code=0 2026-02-24T18:08:31.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2433: main: teardown td/bad-inc-map 0 2026-02-24T18:08:31.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/bad-inc-map 2026-02-24T18:08:31.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-24T18:08:31.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/bad-inc-map KILL 2026-02-24T18:08:31.530 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:08:31.530 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:08:31.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:08:31.531 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:08:31.531 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:08:31.531 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:08:31.532 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:08:31.532 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:08:31.533 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:08:31.534 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:08:31.534 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:08:31.535 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:08:31.536 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:08:31.536 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:08:31.536 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:08:31.537 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:08:31.537 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:08:31.538 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-24T18:08:31.538 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/bad-inc-map 2026-02-24T18:08:31.539 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:08:31.540 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:31.540 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11451 2026-02-24T18:08:31.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11451 2026-02-24T18:08:31.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:08:31.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:08:31.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2434: main: return 0 2026-02-24T18:08:31.542 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T18:08:31.542 DEBUG:teuthology.orchestra.run.trial172:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T18:08:31.594 INFO:tasks.workunit:Running workunit osd/divergent-priors.sh... 2026-02-24T18:08:31.594 DEBUG:teuthology.orchestra.run.trial172:workunit test osd/divergent-priors.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=618c318a7a0b287ab5aeff207b7bf1d73ef45044 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh 2026-02-24T18:08:31.647 INFO:tasks.workunit.client.0.trial172.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-24T18:08:31.650 INFO:tasks.workunit.client.0.trial172.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-24T18:08:31.650 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T18:08:31.650 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T18:08:31.650 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: export PYTHONWARNINGS=ignore 2026-02-24T18:08:31.650 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: PYTHONWARNINGS=ignore 2026-02-24T18:08:31.650 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: export CEPH_CONF=/dev/null 2026-02-24T18:08:31.651 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: CEPH_CONF=/dev/null 2026-02-24T18:08:31.651 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: main: unset CEPH_ARGS 2026-02-24T18:08:31.651 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: main: local code 2026-02-24T18:08:31.651 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2428: main: run td/divergent-priors 2026-02-24T18:08:31.651 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:21: run: local dir=td/divergent-priors 2026-02-24T18:08:31.651 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:22: run: shift 2026-02-24T18:08:31.651 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:25: run: export loglen=12 2026-02-24T18:08:31.651 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:25: run: loglen=12 2026-02-24T18:08:31.651 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:26: run: export divisor=3 2026-02-24T18:08:31.651 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:26: run: divisor=3 2026-02-24T18:08:31.651 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: expr 12 / 2 2026-02-24T18:08:31.652 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: export trim=6 2026-02-24T18:08:31.652 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: trim=6 2026-02-24T18:08:31.652 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: expr 6 / 3 2026-02-24T18:08:31.653 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: export DIVERGENT_WRITE=2 2026-02-24T18:08:31.653 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: DIVERGENT_WRITE=2 2026-02-24T18:08:31.653 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: expr 6 / 3 2026-02-24T18:08:31.655 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: export DIVERGENT_REMOVE=2 2026-02-24T18:08:31.655 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: DIVERGENT_REMOVE=2 2026-02-24T18:08:31.655 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: expr 6 / 3 2026-02-24T18:08:31.657 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: export DIVERGENT_CREATE=2 2026-02-24T18:08:31.657 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: DIVERGENT_CREATE=2 2026-02-24T18:08:31.657 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:31: run: export poolname=test 2026-02-24T18:08:31.657 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:31: run: poolname=test 2026-02-24T18:08:31.657 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:32: run: export testobjects=100 2026-02-24T18:08:31.657 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:32: run: testobjects=100 2026-02-24T18:08:31.657 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:34: run: export CEPH_MON=127.0.0.1:7115 2026-02-24T18:08:31.657 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:34: run: CEPH_MON=127.0.0.1:7115 2026-02-24T18:08:31.657 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:35: run: export CEPH_ARGS 2026-02-24T18:08:31.657 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:36: run: uuidgen 2026-02-24T18:08:31.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:36: run: CEPH_ARGS+='--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none ' 2026-02-24T18:08:31.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:37: run: CEPH_ARGS+='--mon-host=127.0.0.1:7115 ' 2026-02-24T18:08:31.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:39: run: CEPH_ARGS+='--osd_force_auth_primary_missing_objects=1000000 ' 2026-02-24T18:08:31.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:40: run: CEPH_ARGS+='--osd_debug_pg_log_writeout=true ' 2026-02-24T18:08:31.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:41: run: CEPH_ARGS+='--osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:08:31.659 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: set 2026-02-24T18:08:31.659 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-24T18:08:31.661 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: local 'funcs=TEST_divergent 2026-02-24T18:08:31.662 INFO:tasks.workunit.client.0.trial172.stderr:TEST_divergent_2 2026-02-24T18:08:31.662 INFO:tasks.workunit.client.0.trial172.stderr:TEST_divergent_3 2026-02-24T18:08:31.662 INFO:tasks.workunit.client.0.trial172.stderr:TEST_divergent_ec' 2026-02-24T18:08:31.662 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-24T18:08:31.662 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-24T18:08:31.662 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/divergent-priors 2026-02-24T18:08:31.662 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/divergent-priors 2026-02-24T18:08:31.662 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-24T18:08:31.662 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:08:31.662 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-24T18:08:31.662 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:08:31.662 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:08:31.663 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:08:31.663 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:08:31.663 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:08:31.664 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:08:31.664 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:08:31.665 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:08:31.666 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:08:31.666 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:08:31.667 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:08:31.667 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:08:31.668 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:08:31.668 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:08:31.668 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:08:31.668 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:08:31.669 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:08:31.671 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:08:31.671 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-24T18:08:31.672 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:08:31.672 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:31.672 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:08:31.672 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19163 2026-02-24T18:08:31.674 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:08:31.674 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:08:31.674 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/divergent-priors 2026-02-24T18:08:31.675 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T18:08:31.675 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:31.675 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:08:31.676 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19163 2026-02-24T18:08:31.677 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T18:08:31.678 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-24T18:08:31.678 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-24T18:08:31.678 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T18:08:31.678 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-24T18:08:31.678 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent td/divergent-priors 2026-02-24T18:08:31.678 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:57: TEST_divergent: local dir=td/divergent-priors 2026-02-24T18:08:31.678 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:59: TEST_divergent: file_with_random_data 2026-02-24T18:08:31.678 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: file_with_random_data: local size_bytes=512 2026-02-24T18:08:31.679 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: file_with_random_data: mktemp 2026-02-24T18:08:31.680 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: file_with_random_data: local file=/tmp/tmp.DTFILqR5zu 2026-02-24T18:08:31.680 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2374: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.DTFILqR5zu bs=512 count=1 2026-02-24T18:08:31.681 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records in 2026-02-24T18:08:31.681 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records out 2026-02-24T18:08:31.681 INFO:tasks.workunit.client.0.trial172.stderr:512 bytes copied, 8.6571e-05 s, 5.9 MB/s 2026-02-24T18:08:31.681 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: file_with_random_data: printf %s /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:31.681 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:59: TEST_divergent: local dummyfile=/tmp/tmp.DTFILqR5zu 2026-02-24T18:08:31.681 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:60: TEST_divergent: local num_osds=3 2026-02-24T18:08:31.682 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:61: TEST_divergent: expr 3 - 1 2026-02-24T18:08:31.683 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:61: TEST_divergent: seq 0 2 2026-02-24T18:08:31.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:61: TEST_divergent: local 'osds=0 2026-02-24T18:08:31.684 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:08:31.684 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:08:31.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:62: TEST_divergent: run_mon td/divergent-priors a 2026-02-24T18:08:31.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/divergent-priors 2026-02-24T18:08:31.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T18:08:31.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T18:08:31.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T18:08:31.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/divergent-priors/a 2026-02-24T18:08:31.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-24T18:08:31.717 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T18:08:31.717 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:08:31.717 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:08:31.717 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:08:31.717 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:31.717 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:08:31.717 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:08:31.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T18:08:31.748 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T18:08:31.750 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T18:08:31.750 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:08:31.750 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:08:31.750 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T18:08:31.750 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T18:08:31.751 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:08:31.751 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:08:31.751 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:08:31.751 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:08:31.751 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:31.751 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:08:31.751 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-mon.a.asok 2026-02-24T18:08:31.751 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:08:31.752 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19163/ceph-mon.a.asok config get fsid 2026-02-24T18:08:31.805 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T18:08:31.806 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:08:31.806 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:08:31.806 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T18:08:31.806 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T18:08:31.806 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:08:31.806 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:08:31.806 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:08:31.806 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:08:31.806 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:31.806 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:08:31.807 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-mon.a.asok 2026-02-24T18:08:31.807 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:08:31.807 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19163/ceph-mon.a.asok config get mon_host 2026-02-24T18:08:31.859 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:63: TEST_divergent: run_mgr td/divergent-priors x 2026-02-24T18:08:31.860 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/divergent-priors 2026-02-24T18:08:31.860 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T18:08:31.860 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T18:08:31.860 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T18:08:31.860 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/divergent-priors/x 2026-02-24T18:08:31.860 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T18:08:31.975 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T18:08:31.975 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:08:31.976 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:08:31.976 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:08:31.976 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:31.976 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:08:31.976 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:08:31.977 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:08:31.977 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:08:32.000 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: for i in $osds 2026-02-24T18:08:32.000 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:66: TEST_divergent: run_osd td/divergent-priors 0 2026-02-24T18:08:32.000 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-24T18:08:32.000 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:08:32.000 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T18:08:32.000 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:08:32.000 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/0 2026-02-24T18:08:32.000 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:08:32.000 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:08:32.001 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:08:32.001 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:08:32.001 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-24T18:08:32.001 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-24T18:08:32.001 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:08:32.001 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:08:32.001 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:08:32.001 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:08:32.001 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:08:32.001 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:08:32.001 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:08:32.001 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:32.001 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:08:32.002 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:08:32.002 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:08:32.002 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:08:32.002 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:08:32.002 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:08:32.002 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:08:32.002 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:08:32.002 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:08:32.002 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:08:32.002 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:08:32.002 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:08:32.002 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:08:32.003 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:08:32.003 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/0 2026-02-24T18:08:32.003 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:08:32.004 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b38c0560-5794-43f5-b966-bdacdd9b4be4 2026-02-24T18:08:32.004 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b38c0560-5794-43f5-b966-bdacdd9b4be4' 2026-02-24T18:08:32.004 INFO:tasks.workunit.client.0.trial172.stdout:add osd0 b38c0560-5794-43f5-b966-bdacdd9b4be4 2026-02-24T18:08:32.004 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:08:32.015 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAg6Z1psyHTABAAk7x9ZTDi5yKw9QiOTWf6VQ== 2026-02-24T18:08:32.015 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAg6Z1psyHTABAAk7x9ZTDi5yKw9QiOTWf6VQ=="}' 2026-02-24T18:08:32.015 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b38c0560-5794-43f5-b966-bdacdd9b4be4 -i td/divergent-priors/0/new.json 2026-02-24T18:08:32.135 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:08:32.143 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/0/new.json 2026-02-24T18:08:32.144 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAg6Z1psyHTABAAk7x9ZTDi5yKw9QiOTWf6VQ== --osd-uuid b38c0560-5794-43f5-b966-bdacdd9b4be4 2026-02-24T18:08:32.163 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:32.161+0000 7fabb70cfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:32.165 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:32.161+0000 7fabb70cfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:32.165 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:32.165+0000 7fabb70cfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:32.166 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:32.165+0000 7fabb70cfa40 -1 bdev(0x555f41d28400 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-24T18:08:32.166 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:32.165+0000 7fabb70cfa40 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-24T18:08:32.915 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-24T18:08:32.915 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:08:32.917 INFO:tasks.workunit.client.0.trial172.stdout:adding osd0 key to auth repository 2026-02-24T18:08:32.917 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T18:08:32.917 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:08:33.038 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:08:33.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T18:08:33.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:08:33.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:08:33.040 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:08:33.040 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:08:33.068 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:33.065+0000 7fdc763aca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:33.070 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:33.069+0000 7fdc763aca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:33.072 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:33.069+0000 7fdc763aca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:33.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T18:08:33.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:08:33.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:08:33.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:08:33.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:08:33.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:33.152 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:08:33.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:08:33.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:33.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:08:33.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:08:33.627 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:33.625+0000 7fdc763aca40 -1 Falling back to public interface 2026-02-24T18:08:34.262 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:08:34.263 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:34.263 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:08:34.264 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:08:34.264 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:34.264 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:08:34.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:08:34.567 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:34.565+0000 7fdc763aca40 -1 osd.0 0 log_to_monitors true 2026-02-24T18:08:35.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:08:35.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:35.374 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:08:35.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:08:35.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:35.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:08:35.589 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/955620619,v1:127.0.0.1:6801/955620619] [v2:127.0.0.1:6802/955620619,v1:127.0.0.1:6803/955620619] exists,up b38c0560-5794-43f5-b966-bdacdd9b4be4 2026-02-24T18:08:35.593 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:08:35.593 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:08:35.593 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:08:35.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: for i in $osds 2026-02-24T18:08:35.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:66: TEST_divergent: run_osd td/divergent-priors 1 2026-02-24T18:08:35.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-24T18:08:35.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:08:35.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T18:08:35.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:08:35.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/1 2026-02-24T18:08:35.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:08:35.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:08:35.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:08:35.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:08:35.595 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-24T18:08:35.595 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-24T18:08:35.595 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:08:35.595 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:08:35.595 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:08:35.595 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:08:35.595 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:08:35.595 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:08:35.595 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:08:35.595 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:35.595 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:08:35.596 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:08:35.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:08:35.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:08:35.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:08:35.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:08:35.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:08:35.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:08:35.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:08:35.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:08:35.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:08:35.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:08:35.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:08:35.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:08:35.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/1 2026-02-24T18:08:35.597 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:08:35.598 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=65c8d646-b403-4a57-bc5c-0e3cbc649eb7 2026-02-24T18:08:35.598 INFO:tasks.workunit.client.0.trial172.stdout:add osd1 65c8d646-b403-4a57-bc5c-0e3cbc649eb7 2026-02-24T18:08:35.598 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 65c8d646-b403-4a57-bc5c-0e3cbc649eb7' 2026-02-24T18:08:35.598 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:08:35.610 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAj6Z1pAkRDJBAAMtEcnhz4jLakI3I0wFNc8Q== 2026-02-24T18:08:35.610 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAj6Z1pAkRDJBAAMtEcnhz4jLakI3I0wFNc8Q=="}' 2026-02-24T18:08:35.610 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 65c8d646-b403-4a57-bc5c-0e3cbc649eb7 -i td/divergent-priors/1/new.json 2026-02-24T18:08:35.829 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:08:35.838 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/1/new.json 2026-02-24T18:08:35.839 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAj6Z1pAkRDJBAAMtEcnhz4jLakI3I0wFNc8Q== --osd-uuid 65c8d646-b403-4a57-bc5c-0e3cbc649eb7 2026-02-24T18:08:35.859 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:35.857+0000 7ffba5e29a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:35.860 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:35.857+0000 7ffba5e29a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:35.861 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:35.861+0000 7ffba5e29a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:35.862 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:35.861+0000 7ffba5e29a40 -1 bdev(0x55cd6376e400 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-24T18:08:35.862 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:35.861+0000 7ffba5e29a40 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-24T18:08:36.524 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-24T18:08:36.525 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:08:36.525 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T18:08:36.526 INFO:tasks.workunit.client.0.trial172.stdout:adding osd1 key to auth repository 2026-02-24T18:08:36.526 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:08:36.812 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:08:36.813 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T18:08:36.813 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:08:36.814 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:08:36.814 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:08:36.814 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:08:36.841 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:36.837+0000 7f473bc26a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:36.843 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:36.841+0000 7f473bc26a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:36.844 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:36.841+0000 7f473bc26a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:37.036 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T18:08:37.036 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:08:37.036 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:08:37.036 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:08:37.036 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:08:37.036 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:37.036 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:08:37.036 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:08:37.036 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:37.036 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:08:37.254 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:08:37.271 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:37.269+0000 7f473bc26a40 -1 Falling back to public interface 2026-02-24T18:08:37.873 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:37.869+0000 7f473bc26a40 -1 osd.1 0 log_to_monitors true 2026-02-24T18:08:38.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:08:38.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:38.257 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:08:38.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:08:38.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:38.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:08:38.477 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:08:39.479 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:08:39.479 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:39.479 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:08:39.479 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:08:39.479 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:39.479 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:08:39.696 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2955788417,v1:127.0.0.1:6811/2955788417] [v2:127.0.0.1:6812/2955788417,v1:127.0.0.1:6813/2955788417] exists,up 65c8d646-b403-4a57-bc5c-0e3cbc649eb7 2026-02-24T18:08:39.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:08:39.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:08:39.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:08:39.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: for i in $osds 2026-02-24T18:08:39.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:66: TEST_divergent: run_osd td/divergent-priors 2 2026-02-24T18:08:39.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-24T18:08:39.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:08:39.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T18:08:39.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:08:39.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/2 2026-02-24T18:08:39.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:08:39.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:08:39.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:08:39.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:08:39.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-24T18:08:39.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-24T18:08:39.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:08:39.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:08:39.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:08:39.698 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:08:39.698 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:08:39.698 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:08:39.698 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/2 2026-02-24T18:08:39.699 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:08:39.700 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2df2f4e3-421e-48b6-b431-4981b05cb657 2026-02-24T18:08:39.700 INFO:tasks.workunit.client.0.trial172.stdout:add osd2 2df2f4e3-421e-48b6-b431-4981b05cb657 2026-02-24T18:08:39.700 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 2df2f4e3-421e-48b6-b431-4981b05cb657' 2026-02-24T18:08:39.701 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:08:39.715 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAn6Z1pGNOPKhAAh6l3Ls5ZroyvfLb9UfhNSA== 2026-02-24T18:08:39.715 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAn6Z1pGNOPKhAAh6l3Ls5ZroyvfLb9UfhNSA=="}' 2026-02-24T18:08:39.715 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2df2f4e3-421e-48b6-b431-4981b05cb657 -i td/divergent-priors/2/new.json 2026-02-24T18:08:39.933 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:08:39.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/2/new.json 2026-02-24T18:08:39.943 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAn6Z1pGNOPKhAAh6l3Ls5ZroyvfLb9UfhNSA== --osd-uuid 2df2f4e3-421e-48b6-b431-4981b05cb657 2026-02-24T18:08:39.962 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:39.961+0000 7f9d9c6eaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:39.964 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:39.961+0000 7f9d9c6eaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:39.965 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:39.961+0000 7f9d9c6eaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:39.965 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:39.961+0000 7f9d9c6eaa40 -1 bdev(0x55e48d26a400 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-24T18:08:39.966 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:39.965+0000 7f9d9c6eaa40 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-24T18:08:40.580 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-24T18:08:40.581 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:08:40.581 INFO:tasks.workunit.client.0.trial172.stdout:adding osd2 key to auth repository 2026-02-24T18:08:40.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T18:08:40.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:08:40.887 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:08:40.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T18:08:40.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:08:40.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:08:40.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:08:40.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:08:40.906 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:40.905+0000 7f4c5efe9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:40.907 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:40.905+0000 7f4c5efe9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:40.908 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:40.905+0000 7f4c5efe9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:08:41.109 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T18:08:41.109 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:08:41.109 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:08:41.109 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:08:41.109 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:08:41.109 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:41.110 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:08:41.110 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:08:41.110 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:41.111 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:08:41.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:08:41.367 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:41.365+0000 7f4c5efe9a40 -1 Falling back to public interface 2026-02-24T18:08:41.900 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:08:41.897+0000 7f4c5efe9a40 -1 osd.2 0 log_to_monitors true 2026-02-24T18:08:42.326 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:08:42.327 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:42.327 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:08:42.328 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:08:42.328 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:42.329 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:08:42.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:08:43.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:08:43.542 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:08:43.542 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:08:43.543 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:08:43.543 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:08:43.543 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:08:43.752 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1535346934,v1:127.0.0.1:6819/1535346934] [v2:127.0.0.1:6820/1535346934,v1:127.0.0.1:6821/1535346934] exists,up 2df2f4e3-421e-48b6-b431-4981b05cb657 2026-02-24T18:08:43.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:08:43.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:08:43.753 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:08:43.753 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:69: TEST_divergent: ceph osd set noout 2026-02-24T18:08:44.045 INFO:tasks.workunit.client.0.trial172.stderr:noout is set 2026-02-24T18:08:44.054 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:70: TEST_divergent: ceph osd set noin 2026-02-24T18:08:44.348 INFO:tasks.workunit.client.0.trial172.stderr:noin is set 2026-02-24T18:08:44.359 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:71: TEST_divergent: ceph osd set nodown 2026-02-24T18:08:44.653 INFO:tasks.workunit.client.0.trial172.stderr:nodown is set 2026-02-24T18:08:44.664 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:72: TEST_divergent: create_pool test 1 1 2026-02-24T18:08:44.664 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-24T18:08:44.908 INFO:tasks.workunit.client.0.trial172.stderr:pool 'test' already exists 2026-02-24T18:08:44.917 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T18:08:45.918 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:73: TEST_divergent: ceph osd pool set test size 3 2026-02-24T18:08:46.235 INFO:tasks.workunit.client.0.trial172.stderr:set pool 1 size to 3 2026-02-24T18:08:46.250 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:74: TEST_divergent: ceph osd pool set test min_size 2 2026-02-24T18:08:46.544 INFO:tasks.workunit.client.0.trial172.stderr:set pool 1 min_size to 2 2026-02-24T18:08:46.556 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:76: TEST_divergent: flush_pg_stats 2026-02-24T18:08:46.556 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:08:46.557 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:08:46.771 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:08:46.771 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:08:46.771 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:08:46.771 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:08:46.771 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:08:46.771 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:08:46.856 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836485 2026-02-24T18:08:46.856 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836485 2026-02-24T18:08:46.856 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836485' 2026-02-24T18:08:46.856 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:08:46.857 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:08:46.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705668 2026-02-24T18:08:46.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705668 2026-02-24T18:08:46.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836485 1-38654705668' 2026-02-24T18:08:46.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:08:46.938 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:08:47.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574851 2026-02-24T18:08:47.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574851 2026-02-24T18:08:47.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836485 1-38654705668 2-55834574851' 2026-02-24T18:08:47.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:08:47.018 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836485 2026-02-24T18:08:47.018 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:08:47.020 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:08:47.020 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836485 2026-02-24T18:08:47.021 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:08:47.022 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836485 2026-02-24T18:08:47.022 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-24T18:08:47.022 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 21474836485 2026-02-24T18:08:47.022 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:08:47.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836483 -lt 21474836485 2026-02-24T18:08:47.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:08:48.237 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:08:48.237 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:08:48.453 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-24T18:08:48.453 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:08:48.454 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705668 2026-02-24T18:08:48.454 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:08:48.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:08:48.456 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705668 2026-02-24T18:08:48.456 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:08:48.458 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705668 2026-02-24T18:08:48.458 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 38654705668 2026-02-24T18:08:48.458 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-24T18:08:48.458 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:08:48.682 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-24T18:08:48.682 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:08:48.682 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574851 2026-02-24T18:08:48.682 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:08:48.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:08:48.685 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574851 2026-02-24T18:08:48.685 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:08:48.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574851 2026-02-24T18:08:48.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-24T18:08:48.686 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 55834574851 2026-02-24T18:08:48.686 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:08:48.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574852 -lt 55834574851 2026-02-24T18:08:48.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:77: TEST_divergent: wait_for_clean 2026-02-24T18:08:48.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:08:48.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:08:48.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:08:48.899 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T18:08:48.900 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:08:48.900 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:08:48.901 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:08:48.901 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:08:48.901 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:08:48.998 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T18:08:48.998 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:08:48.998 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:08:48.998 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:08:48.998 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:08:48.998 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:08:49.227 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:08:49.227 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:08:49.227 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:08:49.227 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:08:49.227 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:08:49.228 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:08:49.308 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836487 2026-02-24T18:08:49.308 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836487 2026-02-24T18:08:49.308 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836487' 2026-02-24T18:08:49.308 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:08:49.308 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:08:49.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705671 2026-02-24T18:08:49.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705671 2026-02-24T18:08:49.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836487 1-38654705671' 2026-02-24T18:08:49.395 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:08:49.395 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:08:49.478 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574854 2026-02-24T18:08:49.478 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574854 2026-02-24T18:08:49.478 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836487 1-38654705671 2-55834574854' 2026-02-24T18:08:49.478 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:08:49.479 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836487 2026-02-24T18:08:49.479 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:08:49.479 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:08:49.480 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836487 2026-02-24T18:08:49.480 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:08:49.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836487 2026-02-24T18:08:49.482 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-24T18:08:49.482 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 21474836487 2026-02-24T18:08:49.482 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:08:49.707 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-24T18:08:49.707 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:08:50.708 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:08:50.709 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:08:50.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-24T18:08:50.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:08:50.934 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705671 2026-02-24T18:08:50.934 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:08:50.935 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:08:50.936 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705671 2026-02-24T18:08:50.936 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:08:50.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705671 2026-02-24T18:08:50.938 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 38654705671 2026-02-24T18:08:50.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-24T18:08:50.938 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:08:51.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705671 -lt 38654705671 2026-02-24T18:08:51.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:08:51.154 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574854 2026-02-24T18:08:51.154 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:08:51.155 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:08:51.155 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574854 2026-02-24T18:08:51.155 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:08:51.157 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574854 2026-02-24T18:08:51.157 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574854' 2026-02-24T18:08:51.157 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 55834574854 2026-02-24T18:08:51.158 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:08:51.384 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-24T18:08:51.384 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:08:51.385 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:08:51.385 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:08:51.681 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:08:51.681 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:08:51.682 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:08:51.682 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:08:51.682 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:08:51.682 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:08:51.682 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:08:51.682 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:08:51.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-24T18:08:51.899 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:08:51.899 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:08:51.899 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:08:52.190 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-24T18:08:52.190 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T18:08:52.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T18:08:52.191 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:80: TEST_divergent: ceph pg dump pgs --format=json 2026-02-24T18:08:52.191 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:80: TEST_divergent: jq '.pg_stats[0].up_primary' 2026-02-24T18:08:52.403 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:08:52.413 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:80: TEST_divergent: local divergent=1 2026-02-24T18:08:52.413 INFO:tasks.workunit.client.0.trial172.stdout:primary and soon to be divergent is 1 2026-02-24T18:08:52.413 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:81: TEST_divergent: echo 'primary and soon to be divergent is 1' 2026-02-24T18:08:52.413 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:82: TEST_divergent: ceph pg dump pgs 2026-02-24T18:08:52.616 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:08:52.616 INFO:tasks.workunit.client.0.trial172.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T18:08:45.800383+0000 0'0 26:22 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:08:44.855101+0000 0'0 2026-02-24T18:08:44.855101+0000 0 0 periodic scrub scheduled @ 2026-02-26T05:44:03.735172+0000 0 0 2026-02-24T18:08:52.616 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:08:52.616 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:08:52.616 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:08:52.625 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:83: TEST_divergent: local non_divergent= 2026-02-24T18:08:52.625 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:84: TEST_divergent: for i in $osds 2026-02-24T18:08:52.625 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:86: TEST_divergent: '[' 0 = 1 ']' 2026-02-24T18:08:52.625 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: non_divergent=' 0' 2026-02-24T18:08:52.626 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:84: TEST_divergent: for i in $osds 2026-02-24T18:08:52.626 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:86: TEST_divergent: '[' 1 = 1 ']' 2026-02-24T18:08:52.626 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:87: TEST_divergent: continue 2026-02-24T18:08:52.626 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:84: TEST_divergent: for i in $osds 2026-02-24T18:08:52.626 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:86: TEST_divergent: '[' 2 = 1 ']' 2026-02-24T18:08:52.626 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: non_divergent=' 0 2' 2026-02-24T18:08:52.626 INFO:tasks.workunit.client.0.trial172.stdout:writing initial objects 2026-02-24T18:08:52.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:92: TEST_divergent: echo 'writing initial objects' 2026-02-24T18:08:52.627 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: seq 1 100 2026-02-24T18:08:52.628 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:52.628 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_1 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:52.649 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:52.649 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_2 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:52.679 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:52.679 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_3 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:52.708 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:52.708 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_4 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:52.736 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:52.736 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_5 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:52.764 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:52.764 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_6 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:52.792 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:52.793 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_7 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:52.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:52.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_8 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:52.848 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:52.848 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_9 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:52.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:52.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_10 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:52.904 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:52.904 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_11 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:52.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:52.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_12 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:52.960 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:52.960 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_13 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:52.988 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:52.988 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_14 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.016 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.016 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_15 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_16 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.072 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.073 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_17 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_18 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.127 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.127 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_19 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.156 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.157 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_20 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_21 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.212 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.212 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_22 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.239 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.239 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_23 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_24 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.289 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.290 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_25 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.316 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.317 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_26 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.345 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.345 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_27 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.372 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.372 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_28 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.400 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.400 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_29 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.428 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.428 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_30 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.456 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.456 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_31 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.477 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_32 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.505 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.506 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_33 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.531 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.532 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_34 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.560 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.560 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_35 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.588 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.588 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_36 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_37 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.643 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.643 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_38 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.671 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.671 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_39 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_40 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.726 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.726 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_41 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.755 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.755 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_42 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.775 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.775 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_43 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.803 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.803 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_44 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.832 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.832 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_45 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.860 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.860 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_46 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_47 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.916 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.917 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_48 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_49 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_50 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:53.999 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:53.999 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_51 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_52 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_53 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.072 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.072 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_54 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_55 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_56 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.155 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.156 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_57 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.176 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.176 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_58 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.204 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.204 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_59 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.229 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.229 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_60 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_61 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_62 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.312 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.313 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_63 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.339 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.340 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_64 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.367 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.367 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_65 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.396 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.396 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_66 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.425 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_67 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.445 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.445 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_68 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.473 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_69 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.500 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.500 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_70 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.528 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.528 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_71 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.556 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.556 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_72 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.584 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.584 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_73 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_74 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_75 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.668 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.668 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_76 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_77 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.724 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.725 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_78 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_79 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.781 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.781 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_80 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.809 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.809 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_81 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.837 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.837 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_82 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.867 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.867 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_83 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.896 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.896 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_84 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.924 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.925 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_85 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.953 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.953 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_86 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:54.975 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:54.976 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_87 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:55.003 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:55.003 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_88 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:55.024 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:55.025 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_89 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:55.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:55.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_90 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:55.074 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:55.075 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_91 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:55.101 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:55.101 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_92 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:55.131 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:55.131 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_93 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:55.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:55.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_94 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:55.173 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:55.173 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_95 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:55.202 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:55.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_96 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:55.229 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:55.229 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_97 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:55.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:55.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_98 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:55.286 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:55.286 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_99 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:55.316 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-24T18:08:55.316 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_100 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:55.344 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-24T18:08:55.345 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: wait_for_clean 2026-02-24T18:08:55.345 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:08:55.345 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:08:55.345 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:08:55.346 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-24T18:08:55.346 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:08:55.347 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:08:55.347 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:08:55.347 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:08:55.347 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:08:55.414 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-24T18:08:55.415 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:08:55.415 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:08:55.415 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:08:55.415 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:08:55.415 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:08:55.629 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:08:55.630 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:08:55.630 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:08:55.630 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:08:55.630 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:08:55.630 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:08:55.714 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836490 2026-02-24T18:08:55.715 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836490 2026-02-24T18:08:55.715 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490' 2026-02-24T18:08:55.715 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:08:55.715 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:08:55.795 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705674 2026-02-24T18:08:55.795 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705674 2026-02-24T18:08:55.795 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-38654705674' 2026-02-24T18:08:55.795 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:08:55.795 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:08:55.879 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574857 2026-02-24T18:08:55.879 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574857 2026-02-24T18:08:55.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-38654705674 2-55834574857' 2026-02-24T18:08:55.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:08:55.880 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836490 2026-02-24T18:08:55.880 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:08:55.881 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:08:55.882 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836490 2026-02-24T18:08:55.882 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:08:55.883 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836490 2026-02-24T18:08:55.883 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 21474836490 2026-02-24T18:08:55.884 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-24T18:08:55.884 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:08:56.103 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-24T18:08:56.103 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:08:57.105 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:08:57.106 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:08:57.325 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836491 -lt 21474836490 2026-02-24T18:08:57.325 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:08:57.325 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705674 2026-02-24T18:08:57.326 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:08:57.327 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:08:57.328 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705674 2026-02-24T18:08:57.328 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:08:57.329 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705674 2026-02-24T18:08:57.329 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 38654705674 2026-02-24T18:08:57.329 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-24T18:08:57.329 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:08:57.553 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705674 -lt 38654705674 2026-02-24T18:08:57.553 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:08:57.554 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574857 2026-02-24T18:08:57.554 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:08:57.555 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:08:57.556 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574857 2026-02-24T18:08:57.556 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:08:57.557 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574857 2026-02-24T18:08:57.557 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 55834574857 2026-02-24T18:08:57.558 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-24T18:08:57.558 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:08:57.783 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-24T18:08:57.783 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:08:57.784 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:08:57.784 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:08:58.065 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:08:58.065 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:08:58.065 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:08:58.066 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:08:58.066 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:08:58.066 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:08:58.066 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:08:58.066 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:08:58.279 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-24T18:08:58.280 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:08:58.280 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:08:58.280 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:08:58.563 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-24T18:08:58.563 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T18:08:58.563 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T18:08:58.563 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:101: TEST_divergent: get_pg test existing_1 2026-02-24T18:08:58.563 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=test 2026-02-24T18:08:58.563 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=existing_1 2026-02-24T18:08:58.564 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map test existing_1 2026-02-24T18:08:58.564 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-24T18:08:58.786 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:101: TEST_divergent: local pgid=1.0 2026-02-24T18:08:58.786 INFO:tasks.workunit.client.0.trial172.stdout:blackholing osds 0 2 2026-02-24T18:08:58.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:104: TEST_divergent: echo 'blackholing osds 0 2' 2026-02-24T18:08:58.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:105: TEST_divergent: ceph pg dump pgs 2026-02-24T18:08:58.992 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:08:58.992 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 0 0 0 51200 0 0 16 84 16 active+clean 2026-02-24T18:08:45.800383+0000 26'100 26:122 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:08:44.855101+0000 0'0 2026-02-24T18:08:44.855101+0000 0 0 periodic scrub scheduled @ 2026-02-26T05:44:03.735172+0000 0 0 2026-02-24T18:08:58.992 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:08:58.993 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:08:58.993 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:08:59.003 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:106: TEST_divergent: for i in $non_divergent 2026-02-24T18:08:59.004 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: get_asok_path osd.0 2026-02-24T18:08:59.004 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T18:08:59.004 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T18:08:59.005 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:08:59.005 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:59.005 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:08:59.005 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.0.asok 2026-02-24T18:08:59.005 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: CEPH_ARGS= 2026-02-24T18:08:59.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-24T18:08:59.069 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:08:59.069 INFO:tasks.workunit.client.0.trial172.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-24T18:08:59.069 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:08:59.075 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:106: TEST_divergent: for i in $non_divergent 2026-02-24T18:08:59.075 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: get_asok_path osd.2 2026-02-24T18:08:59.076 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-24T18:08:59.076 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-24T18:08:59.076 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:08:59.076 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:59.076 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:08:59.076 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.2.asok 2026-02-24T18:08:59.076 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: CEPH_ARGS= 2026-02-24T18:08:59.077 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-24T18:08:59.128 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:08:59.128 INFO:tasks.workunit.client.0.trial172.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-24T18:08:59.128 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:08:59.133 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: local case5=100 2026-02-24T18:08:59.134 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:112: TEST_divergent: expr 100 - 1 2026-02-24T18:08:59.135 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:112: TEST_divergent: local case3=99 2026-02-24T18:08:59.135 INFO:tasks.workunit.client.0.trial172.stdout:writing divergent object 2026-02-24T18:08:59.135 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:114: TEST_divergent: echo 'writing divergent object' 2026-02-24T18:08:59.136 INFO:tasks.workunit.client.0.trial172.stdout:create missing divergent object 2026-02-24T18:08:59.137 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:116: TEST_divergent: echo 'create missing divergent object' 2026-02-24T18:08:59.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:117: TEST_divergent: inject_eio rep data test existing_99 td/divergent-priors 0 2026-02-24T18:08:59.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=rep 2026-02-24T18:08:59.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T18:08:59.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T18:08:59.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T18:08:59.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=test 2026-02-24T18:08:59.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:115: TEST_divergent: rados -p test put existing_100 /tmp/tmp.DTFILqR5zu 2026-02-24T18:08:59.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T18:08:59.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=existing_99 2026-02-24T18:08:59.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T18:08:59.139 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/divergent-priors 2026-02-24T18:08:59.139 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T18:08:59.139 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=0 2026-02-24T18:08:59.139 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T18:08:59.139 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds test existing_99 2026-02-24T18:08:59.139 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=test 2026-02-24T18:08:59.139 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=existing_99 2026-02-24T18:08:59.139 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map test existing_99 2026-02-24T18:08:59.139 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T18:08:59.358 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-24T18:08:59.358 INFO:tasks.workunit.client.0.trial172.stderr:0 2026-02-24T18:08:59.358 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:08:59.358 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2 2026-02-24T18:08:59.359 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('1' '0' '2') 2026-02-24T18:08:59.359 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T18:08:59.359 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=1 2026-02-24T18:08:59.359 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' rep '!=' ec ']' 2026-02-24T18:08:59.359 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2525: inject_eio: shard_id= 2026-02-24T18:08:59.359 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/divergent-priors/1/type 2026-02-24T18:08:59.361 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T18:08:59.361 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-24T18:08:59.361 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T18:08:59.361 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-24T18:08:59.361 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T18:08:59.361 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T18:08:59.361 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T18:08:59.362 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-24T18:08:59.362 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T18:08:59.362 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T18:08:59.362 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:08:59.362 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:59.362 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:08:59.362 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.1.asok 2026-02-24T18:08:59.362 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.19163/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-24T18:08:59.421 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T18:08:59.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T18:08:59.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T18:08:59.423 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.1 2026-02-24T18:08:59.423 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T18:08:59.423 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T18:08:59.423 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:08:59.423 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:08:59.424 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:08:59.424 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.1.asok 2026-02-24T18:08:59.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T18:08:59.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.1.asok injectdataerr test existing_99 2026-02-24T18:08:59.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:119: TEST_divergent: sleep 10 2026-02-24T18:08:59.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:118: TEST_divergent: rados -p test get existing_99 td/divergent-priors/existing 2026-02-24T18:09:09.482 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:120: TEST_divergent: killall -9 rados 2026-02-24T18:09:09.523 INFO:tasks.workunit.client.0.trial172.stderr:rados: no process found 2026-02-24T18:09:09.523 INFO:tasks.workunit.client.0.trial172.stdout:killing all the osds 2026-02-24T18:09:09.523 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:123: TEST_divergent: echo 'killing all the osds' 2026-02-24T18:09:09.524 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:124: TEST_divergent: ceph pg dump pgs 2026-02-24T18:09:09.743 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:09:09.743 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 0 0 0 51200 0 0 17 84 17 active+clean 2026-02-24T18:08:59.502985+0000 26'101 26:131 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:08:44.855101+0000 0'0 2026-02-24T18:08:44.855101+0000 0 0 periodic scrub scheduled @ 2026-02-26T05:44:03.735172+0000 0 0 2026-02-24T18:09:09.743 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:09:09.743 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:09:09.743 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:09:09.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:125: TEST_divergent: kill_daemons td/divergent-priors KILL osd 2026-02-24T18:09:09.753 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:09:09.753 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:09:09.753 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:09:09.753 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:09:09.753 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:09:09.862 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:09:09.862 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:126: TEST_divergent: for i in $osds 2026-02-24T18:09:09.862 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:128: TEST_divergent: ceph osd down osd.0 2026-02-24T18:09:10.131 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already down. 2026-02-24T18:09:10.142 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:126: TEST_divergent: for i in $osds 2026-02-24T18:09:10.142 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:128: TEST_divergent: ceph osd down osd.1 2026-02-24T18:09:10.411 INFO:tasks.workunit.client.0.trial172.stderr:osd.1 is already down. 2026-02-24T18:09:10.421 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:126: TEST_divergent: for i in $osds 2026-02-24T18:09:10.421 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:128: TEST_divergent: ceph osd down osd.2 2026-02-24T18:09:10.699 INFO:tasks.workunit.client.0.trial172.stderr:osd.2 is already down. 2026-02-24T18:09:10.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:130: TEST_divergent: for i in $non_divergent 2026-02-24T18:09:10.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:132: TEST_divergent: ceph osd out osd.0 2026-02-24T18:09:10.985 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already out. 2026-02-24T18:09:10.994 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:130: TEST_divergent: for i in $non_divergent 2026-02-24T18:09:10.994 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:132: TEST_divergent: ceph osd out osd.2 2026-02-24T18:09:11.269 INFO:tasks.workunit.client.0.trial172.stderr:osd.2 is already out. 2026-02-24T18:09:11.278 INFO:tasks.workunit.client.0.trial172.stdout:bringing up non_divergent 0 2 2026-02-24T18:09:11.279 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:136: TEST_divergent: echo 'bringing up non_divergent 0 2' 2026-02-24T18:09:11.279 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:137: TEST_divergent: ceph pg dump pgs 2026-02-24T18:09:11.496 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:09:11.496 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 0 0 0 51200 0 0 17 84 17 stale+active+clean 2026-02-24T18:08:59.502985+0000 26'101 26:131 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:08:44.855101+0000 0'0 2026-02-24T18:08:44.855101+0000 0 0 periodic scrub scheduled @ 2026-02-26T05:44:03.735172+0000 0 0 2026-02-24T18:09:11.496 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:09:11.496 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:09:11.497 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:09:11.509 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:138: TEST_divergent: for i in $non_divergent 2026-02-24T18:09:11.509 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:140: TEST_divergent: activate_osd td/divergent-priors 0 2026-02-24T18:09:11.510 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:09:11.510 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:09:11.510 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-24T18:09:11.510 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:09:11.510 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/0 2026-02-24T18:09:11.510 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:09:11.510 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:09:11.510 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:09:11.510 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:09:11.510 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-24T18:09:11.510 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-24T18:09:11.511 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:09:11.511 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:09:11.511 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:09:11.511 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:09:11.511 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:09:11.511 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:09:11.511 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:09:11.511 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:09:11.511 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:09:11.511 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:09:11.511 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:09:11.512 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:09:11.512 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:09:11.512 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:09:11.512 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:09:11.512 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:09:11.512 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:09:11.512 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:09:11.512 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:09:11.512 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:09:11.512 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/0 2026-02-24T18:09:11.513 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:09:11.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-24T18:09:11.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:09:11.514 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/0/whoami 2026-02-24T18:09:11.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-24T18:09:11.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:09:11.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:09:11.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:09:11.533 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:11.528+0000 7fdd3e0ffa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:09:11.534 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:11.532+0000 7fdd3e0ffa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:09:11.535 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:11.532+0000 7fdd3e0ffa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:09:11.734 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-24T18:09:11.734 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:09:11.734 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:09:11.735 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:09:11.735 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:09:11.735 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:09:11.735 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:09:11.736 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:09:11.736 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:09:11.737 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:09:11.903 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:11.900+0000 7fdd3e0ffa40 -1 Falling back to public interface 2026-02-24T18:09:11.957 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:09:12.507 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:12.504+0000 7fdd3e0ffa40 -1 osd.0 26 log_to_monitors true 2026-02-24T18:09:12.959 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:09:12.959 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:09:12.959 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:09:12.960 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:09:12.960 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:09:12.961 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:09:13.179 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up out weight 0 up_from 33 up_thru 0 down_at 27 last_clean_interval [5,26) [v2:127.0.0.1:6800/1337209777,v1:127.0.0.1:6801/1337209777] [v2:127.0.0.1:6802/1337209777,v1:127.0.0.1:6803/1337209777] exists,up b38c0560-5794-43f5-b966-bdacdd9b4be4 2026-02-24T18:09:13.182 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:09:13.182 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:09:13.182 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:09:13.182 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:138: TEST_divergent: for i in $non_divergent 2026-02-24T18:09:13.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:140: TEST_divergent: activate_osd td/divergent-priors 2 2026-02-24T18:09:13.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:09:13.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:09:13.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T18:09:13.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:09:13.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/2 2026-02-24T18:09:13.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:09:13.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:09:13.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:09:13.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:09:13.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-24T18:09:13.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-24T18:09:13.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:09:13.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:09:13.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:09:13.184 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:09:13.184 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:09:13.184 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:09:13.184 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:09:13.184 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:09:13.184 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:09:13.184 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:09:13.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:09:13.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:09:13.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:09:13.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:09:13.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:09:13.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:09:13.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:09:13.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:09:13.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:09:13.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:09:13.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/2 2026-02-24T18:09:13.186 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:09:13.186 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T18:09:13.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:09:13.187 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/2/whoami 2026-02-24T18:09:13.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T18:09:13.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:09:13.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:09:13.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:09:13.199 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:13.196+0000 7f0484440a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:09:13.201 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:13.196+0000 7f0484440a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:09:13.201 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:13.196+0000 7f0484440a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:09:13.410 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T18:09:13.410 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:09:13.410 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:09:13.410 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:09:13.410 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:09:13.410 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:09:13.410 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:09:13.411 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:09:13.411 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:09:13.411 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:09:13.628 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:09:13.639 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:13.636+0000 7f0484440a40 -1 Falling back to public interface 2026-02-24T18:09:14.211 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:14.208+0000 7f0484440a40 -1 osd.2 26 log_to_monitors true 2026-02-24T18:09:14.629 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:09:14.630 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:09:14.630 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:09:14.630 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:09:14.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:09:14.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:09:14.843 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up out weight 0 up_from 35 up_thru 0 down_at 29 last_clean_interval [13,26) [v2:127.0.0.1:6810/2060230136,v1:127.0.0.1:6811/2060230136] [v2:127.0.0.1:6812/2060230136,v1:127.0.0.1:6813/2060230136] exists,up 2df2f4e3-421e-48b6-b431-4981b05cb657 2026-02-24T18:09:14.843 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:09:14.844 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:09:14.844 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:09:14.844 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:142: TEST_divergent: for i in $non_divergent 2026-02-24T18:09:14.844 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:144: TEST_divergent: ceph osd in osd.0 2026-02-24T18:09:15.091 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already in. 2026-02-24T18:09:15.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:142: TEST_divergent: for i in $non_divergent 2026-02-24T18:09:15.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:144: TEST_divergent: ceph osd in osd.2 2026-02-24T18:09:15.369 INFO:tasks.workunit.client.0.trial172.stderr:osd.2 is already in. 2026-02-24T18:09:15.379 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:147: TEST_divergent: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-24T18:09:15.379 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:147: TEST_divergent: wait_for_clean 2026-02-24T18:09:15.380 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:09:15.380 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:09:15.380 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:09:15.380 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-24T18:09:15.380 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:09:15.380 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:09:15.381 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:09:15.381 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:09:15.381 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:09:15.446 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-24T18:09:15.446 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:09:15.446 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:09:15.446 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:09:15.446 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:09:15.446 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:09:15.667 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:09:15.668 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:09:15.668 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:09:15.668 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:09:15.668 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:09:15.668 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:09:15.747 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920771 2026-02-24T18:09:15.748 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920771 2026-02-24T18:09:15.748 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-141733920771' 2026-02-24T18:09:15.748 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:09:15.748 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:09:15.814 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq= 2026-02-24T18:09:15.814 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: true 2026-02-24T18:09:15.814 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z '' 2026-02-24T18:09:15.814 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: flush_pg_stats: continue 2026-02-24T18:09:15.814 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:09:15.814 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:09:15.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=150323855363 2026-02-24T18:09:15.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 150323855363 2026-02-24T18:09:15.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-141733920771 2-150323855363' 2026-02-24T18:09:15.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:09:15.897 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-141733920771 2026-02-24T18:09:15.897 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:09:15.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:09:15.900 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-141733920771 2026-02-24T18:09:15.900 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:09:15.901 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920771 2026-02-24T18:09:15.901 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 141733920771 2026-02-24T18:09:15.902 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 141733920771' 2026-02-24T18:09:15.902 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:09:16.127 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 1 -lt 141733920771 2026-02-24T18:09:16.127 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:09:17.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:09:17.128 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:09:17.350 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920771 -lt 141733920771 2026-02-24T18:09:17.350 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:09:17.350 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-150323855363 2026-02-24T18:09:17.351 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:09:17.352 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:09:17.353 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-150323855363 2026-02-24T18:09:17.353 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:09:17.354 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=150323855363 2026-02-24T18:09:17.354 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 150323855363 2026-02-24T18:09:17.354 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 150323855363' 2026-02-24T18:09:17.354 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:09:17.577 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 150323855363 -lt 150323855363 2026-02-24T18:09:17.578 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:09:17.578 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:09:17.578 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:09:17.859 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:09:17.859 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:09:17.860 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:09:17.860 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:09:17.860 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:09:17.860 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:09:17.860 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:09:17.860 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:09:18.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:09:18.071 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:09:18.071 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:09:18.071 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:09:18.352 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:09:18.352 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-24T18:09:18.353 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-24T18:09:18.353 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-24T18:09:18.353 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:09:18.353 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-24T18:09:18.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:09:18.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:09:18.453 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:09:18.453 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:09:18.453 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:09:18.453 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:09:18.453 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:09:18.453 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:09:18.668 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:09:18.668 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:09:18.669 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:09:18.669 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:09:18.964 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:09:18.964 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:09:18.964 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:09:18.964 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:09:18.964 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:09:18.965 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:09:18.965 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:09:18.965 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:09:18.965 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:09:19.254 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:09:19.255 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:09:19.255 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 8 )) 2026-02-24T18:09:19.255 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:09:19.255 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-24T18:09:19.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:09:19.456 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:09:19.456 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:09:19.457 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:09:19.457 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:09:19.457 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:09:19.457 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:09:19.457 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:09:19.673 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:09:19.674 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:09:19.674 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:09:19.674 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:09:19.964 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:09:19.964 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:09:19.965 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:09:19.965 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:09:19.965 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:09:19.965 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:09:19.965 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:09:19.965 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:09:19.965 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:09:20.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:09:20.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:09:20.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 8 )) 2026-02-24T18:09:20.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:09:20.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-24T18:09:20.662 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:09:20.662 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:09:20.663 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:09:20.664 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:09:20.664 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:09:20.664 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:09:20.664 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:09:20.664 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:09:20.883 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:09:20.883 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:09:20.884 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:09:20.884 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:09:21.175 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:09:21.175 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:09:21.175 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:09:21.175 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:09:21.175 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:09:21.175 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:09:21.175 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:09:21.175 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:09:21.175 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:09:21.468 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:09:21.468 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:09:21.468 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 8 )) 2026-02-24T18:09:21.468 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:09:21.468 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-24T18:09:22.269 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:09:22.269 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:09:22.269 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:09:22.269 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:09:22.269 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:09:22.269 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:09:22.269 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:09:22.270 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:09:22.486 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:09:22.486 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:09:22.486 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:09:22.486 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:09:22.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:09:22.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:09:22.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:09:22.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:09:22.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:09:22.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:09:22.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:09:22.774 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:09:22.775 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:09:23.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:09:23.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:09:23.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 8 )) 2026-02-24T18:09:23.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:09:23.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 1.6 2026-02-24T18:09:24.670 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:09:24.670 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:09:24.670 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:09:24.670 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:09:24.670 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:09:24.670 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:09:24.671 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:09:24.671 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:09:24.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:09:24.889 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:09:24.889 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:09:24.889 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:09:25.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:09:25.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:09:25.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:09:25.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:09:25.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:09:25.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:09:25.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:09:25.179 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:09:25.180 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:09:25.471 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:09:25.471 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:09:25.471 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 5 >= 8 )) 2026-02-24T18:09:25.471 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:09:25.471 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 3.2 2026-02-24T18:09:28.674 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:09:28.674 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:09:28.675 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:09:28.675 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:09:28.675 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:09:28.675 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:09:28.675 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:09:28.675 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:09:28.892 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:09:28.893 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:09:28.893 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:09:28.893 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:09:29.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:09:29.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:09:29.188 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:09:29.188 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:09:29.188 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:09:29.188 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:09:29.188 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:09:29.188 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:09:29.188 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:09:29.480 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:09:29.480 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:09:29.480 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 6 >= 8 )) 2026-02-24T18:09:29.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:09:29.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 6.4 2026-02-24T18:09:35.881 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:09:35.881 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:09:35.882 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:09:35.882 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:09:35.882 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:09:35.882 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:09:35.883 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:09:35.883 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:09:36.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:09:36.102 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:09:36.103 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:09:36.103 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:09:36.389 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:09:36.389 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:09:36.389 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:09:36.389 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:09:36.389 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:09:36.389 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:09:36.389 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:09:36.389 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:09:36.389 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:09:36.671 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:09:36.671 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:09:36.671 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 7 >= 8 )) 2026-02-24T18:09:36.671 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:09:36.671 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 7.3 2026-02-24T18:09:43.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:09:43.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:09:43.974 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:09:43.974 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:09:43.974 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:09:43.974 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:09:43.974 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:09:43.974 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:09:44.182 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:09:44.183 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:09:44.183 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:09:44.183 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:09:44.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:09:44.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:09:44.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:09:44.477 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:09:44.477 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:09:44.477 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:09:44.477 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:09:44.477 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:09:44.477 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:09:44.772 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:09:44.772 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:09:44.772 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 8 >= 8 )) 2026-02-24T18:09:44.772 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-24T18:09:45.045 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:09:45.045 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_fingerprint": "82c82048-fced-4083-b8a5-8d43b7ac206c", 2026-02-24T18:09:45.045 INFO:tasks.workunit.client.0.trial172.stdout: "version": "20.3.0-5507-g618c318a", 2026-02-24T18:09:45.046 INFO:tasks.workunit.client.0.trial172.stdout: "commit": "618c318a7a0b287ab5aeff207b7bf1d73ef45044", 2026-02-24T18:09:45.046 INFO:tasks.workunit.client.0.trial172.stdout: "timestamp": "2026-02-24T18:09:45.043518+0000", 2026-02-24T18:09:45.046 INFO:tasks.workunit.client.0.trial172.stdout: "tag": "", 2026-02-24T18:09:45.046 INFO:tasks.workunit.client.0.trial172.stdout: "health": { 2026-02-24T18:09:45.046 INFO:tasks.workunit.client.0.trial172.stdout: "status": "HEALTH_WARN", 2026-02-24T18:09:45.046 INFO:tasks.workunit.client.0.trial172.stdout: "checks": { 2026-02-24T18:09:45.046 INFO:tasks.workunit.client.0.trial172.stdout: "OSDMAP_FLAGS": { 2026-02-24T18:09:45.046 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:09:45.046 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:09:45.046 INFO:tasks.workunit.client.0.trial172.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-24T18:09:45.046 INFO:tasks.workunit.client.0.trial172.stdout: "count": 17 2026-02-24T18:09:45.047 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.047 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [], 2026-02-24T18:09:45.047 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:09:45.047 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.047 INFO:tasks.workunit.client.0.trial172.stdout: "OSD_DOWN": { 2026-02-24T18:09:45.047 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:09:45.047 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:09:45.047 INFO:tasks.workunit.client.0.trial172.stdout: "message": "1 osds down", 2026-02-24T18:09:45.047 INFO:tasks.workunit.client.0.trial172.stdout: "count": 1 2026-02-24T18:09:45.047 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.047 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [ 2026-02-24T18:09:45.047 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.048 INFO:tasks.workunit.client.0.trial172.stdout: "message": "osd.1 (root=default,host=trial172) is down" 2026-02-24T18:09:45.048 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.048 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.048 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:09:45.048 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.048 INFO:tasks.workunit.client.0.trial172.stdout: "PG_DEGRADED": { 2026-02-24T18:09:45.048 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:09:45.048 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:09:45.048 INFO:tasks.workunit.client.0.trial172.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2026-02-24T18:09:45.048 INFO:tasks.workunit.client.0.trial172.stdout: "count": 1 2026-02-24T18:09:45.048 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.048 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [ 2026-02-24T18:09:45.048 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.049 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:09:45.049 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.049 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.049 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:09:45.049 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.049 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.049 INFO:tasks.workunit.client.0.trial172.stdout: "mutes": [] 2026-02-24T18:09:45.049 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.049 INFO:tasks.workunit.client.0.trial172.stdout: "monmap_first_committed": 1, 2026-02-24T18:09:45.049 INFO:tasks.workunit.client.0.trial172.stdout: "monmap_last_committed": 1, 2026-02-24T18:09:45.049 INFO:tasks.workunit.client.0.trial172.stdout: "monmap": { 2026-02-24T18:09:45.049 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 1, 2026-02-24T18:09:45.050 INFO:tasks.workunit.client.0.trial172.stdout: "fsid": "27512777-e37f-4f16-b176-1621842bd4df", 2026-02-24T18:09:45.050 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:08:31.707419Z", 2026-02-24T18:09:45.050 INFO:tasks.workunit.client.0.trial172.stdout: "created": "2026-02-24T18:08:31.707419Z", 2026-02-24T18:09:45.050 INFO:tasks.workunit.client.0.trial172.stdout: "min_mon_release": 20, 2026-02-24T18:09:45.050 INFO:tasks.workunit.client.0.trial172.stdout: "min_mon_release_name": "tentacle", 2026-02-24T18:09:45.050 INFO:tasks.workunit.client.0.trial172.stdout: "election_strategy": 1, 2026-02-24T18:09:45.050 INFO:tasks.workunit.client.0.trial172.stdout: "disallowed_leaders": "", 2026-02-24T18:09:45.050 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode": false, 2026-02-24T18:09:45.050 INFO:tasks.workunit.client.0.trial172.stdout: "tiebreaker_mon": "", 2026-02-24T18:09:45.050 INFO:tasks.workunit.client.0.trial172.stdout: "removed_ranks": "", 2026-02-24T18:09:45.050 INFO:tasks.workunit.client.0.trial172.stdout: "features": { 2026-02-24T18:09:45.050 INFO:tasks.workunit.client.0.trial172.stdout: "persistent": [ 2026-02-24T18:09:45.051 INFO:tasks.workunit.client.0.trial172.stdout: "kraken", 2026-02-24T18:09:45.051 INFO:tasks.workunit.client.0.trial172.stdout: "luminous", 2026-02-24T18:09:45.051 INFO:tasks.workunit.client.0.trial172.stdout: "mimic", 2026-02-24T18:09:45.051 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap-prune", 2026-02-24T18:09:45.051 INFO:tasks.workunit.client.0.trial172.stdout: "nautilus", 2026-02-24T18:09:45.051 INFO:tasks.workunit.client.0.trial172.stdout: "octopus", 2026-02-24T18:09:45.051 INFO:tasks.workunit.client.0.trial172.stdout: "pacific", 2026-02-24T18:09:45.051 INFO:tasks.workunit.client.0.trial172.stdout: "elector-pinging", 2026-02-24T18:09:45.051 INFO:tasks.workunit.client.0.trial172.stdout: "quincy", 2026-02-24T18:09:45.051 INFO:tasks.workunit.client.0.trial172.stdout: "reef", 2026-02-24T18:09:45.051 INFO:tasks.workunit.client.0.trial172.stdout: "squid", 2026-02-24T18:09:45.051 INFO:tasks.workunit.client.0.trial172.stdout: "tentacle", 2026-02-24T18:09:45.051 INFO:tasks.workunit.client.0.trial172.stdout: "nvmeof_beacon_diff" 2026-02-24T18:09:45.052 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.052 INFO:tasks.workunit.client.0.trial172.stdout: "optional": [] 2026-02-24T18:09:45.052 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.052 INFO:tasks.workunit.client.0.trial172.stdout: "mons": [ 2026-02-24T18:09:45.052 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.052 INFO:tasks.workunit.client.0.trial172.stdout: "rank": 0, 2026-02-24T18:09:45.052 INFO:tasks.workunit.client.0.trial172.stdout: "name": "a", 2026-02-24T18:09:45.052 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:09:45.052 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:09:45.052 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.052 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:09:45.052 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:7115", 2026-02-24T18:09:45.052 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 0 2026-02-24T18:09:45.053 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.053 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.053 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.053 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:7115/0", 2026-02-24T18:09:45.053 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-24T18:09:45.053 INFO:tasks.workunit.client.0.trial172.stdout: "priority": 0, 2026-02-24T18:09:45.053 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 0, 2026-02-24T18:09:45.053 INFO:tasks.workunit.client.0.trial172.stdout: "time_added": "2026-02-24T18:08:31:707417+0000", 2026-02-24T18:09:45.053 INFO:tasks.workunit.client.0.trial172.stdout: "crush_location": [] 2026-02-24T18:09:45.053 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.053 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.053 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.053 INFO:tasks.workunit.client.0.trial172.stdout: "quorum": [ 2026-02-24T18:09:45.054 INFO:tasks.workunit.client.0.trial172.stdout: 0 2026-02-24T18:09:45.054 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.054 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap": { 2026-02-24T18:09:45.054 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 40, 2026-02-24T18:09:45.054 INFO:tasks.workunit.client.0.trial172.stdout: "fsid": "27512777-e37f-4f16-b176-1621842bd4df", 2026-02-24T18:09:45.054 INFO:tasks.workunit.client.0.trial172.stdout: "created": "2026-02-24T18:08:31.790674+0000", 2026-02-24T18:09:45.054 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:09:19.526008+0000", 2026-02-24T18:09:45.054 INFO:tasks.workunit.client.0.trial172.stdout: "last_up_change": "2026-02-24T18:09:14.401045+0000", 2026-02-24T18:09:45.054 INFO:tasks.workunit.client.0.trial172.stdout: "last_in_change": "2026-02-24T18:09:15.311339+0000", 2026-02-24T18:09:45.054 INFO:tasks.workunit.client.0.trial172.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-24T18:09:45.054 INFO:tasks.workunit.client.0.trial172.stdout: "flags_num": 5800384, 2026-02-24T18:09:45.054 INFO:tasks.workunit.client.0.trial172.stdout: "flags_set": [ 2026-02-24T18:09:45.054 INFO:tasks.workunit.client.0.trial172.stdout: "nodown", 2026-02-24T18:09:45.055 INFO:tasks.workunit.client.0.trial172.stdout: "noin", 2026-02-24T18:09:45.055 INFO:tasks.workunit.client.0.trial172.stdout: "noout", 2026-02-24T18:09:45.055 INFO:tasks.workunit.client.0.trial172.stdout: "pglog_hardlimit", 2026-02-24T18:09:45.055 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snapdirs", 2026-02-24T18:09:45.055 INFO:tasks.workunit.client.0.trial172.stdout: "recovery_deletes", 2026-02-24T18:09:45.055 INFO:tasks.workunit.client.0.trial172.stdout: "sortbitwise" 2026-02-24T18:09:45.055 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.055 INFO:tasks.workunit.client.0.trial172.stdout: "crush_version": 9, 2026-02-24T18:09:45.055 INFO:tasks.workunit.client.0.trial172.stdout: "full_ratio": 0.99000000953674316, 2026-02-24T18:09:45.055 INFO:tasks.workunit.client.0.trial172.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-24T18:09:45.055 INFO:tasks.workunit.client.0.trial172.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-24T18:09:45.055 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_snapshot": "", 2026-02-24T18:09:45.055 INFO:tasks.workunit.client.0.trial172.stdout: "pool_max": 1, 2026-02-24T18:09:45.056 INFO:tasks.workunit.client.0.trial172.stdout: "max_osd": 3, 2026-02-24T18:09:45.056 INFO:tasks.workunit.client.0.trial172.stdout: "require_min_compat_client": "luminous", 2026-02-24T18:09:45.056 INFO:tasks.workunit.client.0.trial172.stdout: "min_compat_client": "jewel", 2026-02-24T18:09:45.056 INFO:tasks.workunit.client.0.trial172.stdout: "require_osd_release": "tentacle", 2026-02-24T18:09:45.056 INFO:tasks.workunit.client.0.trial172.stdout: "allow_crimson": false, 2026-02-24T18:09:45.056 INFO:tasks.workunit.client.0.trial172.stdout: "pools": [ 2026-02-24T18:09:45.056 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.056 INFO:tasks.workunit.client.0.trial172.stdout: "pool": 1, 2026-02-24T18:09:45.056 INFO:tasks.workunit.client.0.trial172.stdout: "pool_name": "test", 2026-02-24T18:09:45.056 INFO:tasks.workunit.client.0.trial172.stdout: "create_time": "2026-02-24T18:08:44.818387+0000", 2026-02-24T18:09:45.056 INFO:tasks.workunit.client.0.trial172.stdout: "flags": 1, 2026-02-24T18:09:45.056 INFO:tasks.workunit.client.0.trial172.stdout: "flags_names": "hashpspool", 2026-02-24T18:09:45.056 INFO:tasks.workunit.client.0.trial172.stdout: "type": 1, 2026-02-24T18:09:45.057 INFO:tasks.workunit.client.0.trial172.stdout: "size": 3, 2026-02-24T18:09:45.057 INFO:tasks.workunit.client.0.trial172.stdout: "min_size": 2, 2026-02-24T18:09:45.057 INFO:tasks.workunit.client.0.trial172.stdout: "crush_rule": 0, 2026-02-24T18:09:45.057 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_count": 0, 2026-02-24T18:09:45.057 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_target": 0, 2026-02-24T18:09:45.057 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_barrier": 0, 2026-02-24T18:09:45.057 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-24T18:09:45.057 INFO:tasks.workunit.client.0.trial172.stdout: "is_stretch_pool": false, 2026-02-24T18:09:45.057 INFO:tasks.workunit.client.0.trial172.stdout: "object_hash": 2, 2026-02-24T18:09:45.057 INFO:tasks.workunit.client.0.trial172.stdout: "pg_autoscale_mode": "off", 2026-02-24T18:09:45.057 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num": 1, 2026-02-24T18:09:45.057 INFO:tasks.workunit.client.0.trial172.stdout: "pg_placement_num": 1, 2026-02-24T18:09:45.057 INFO:tasks.workunit.client.0.trial172.stdout: "pg_placement_num_target": 1, 2026-02-24T18:09:45.058 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num_target": 1, 2026-02-24T18:09:45.058 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num_pending": 1, 2026-02-24T18:09:45.058 INFO:tasks.workunit.client.0.trial172.stdout: "last_pg_merge_meta": { 2026-02-24T18:09:45.058 INFO:tasks.workunit.client.0.trial172.stdout: "source_pgid": "0.0", 2026-02-24T18:09:45.058 INFO:tasks.workunit.client.0.trial172.stdout: "ready_epoch": 0, 2026-02-24T18:09:45.058 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_started": 0, 2026-02-24T18:09:45.058 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_clean": 0, 2026-02-24T18:09:45.058 INFO:tasks.workunit.client.0.trial172.stdout: "source_version": "0'0", 2026-02-24T18:09:45.058 INFO:tasks.workunit.client.0.trial172.stdout: "target_version": "0'0" 2026-02-24T18:09:45.058 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.058 INFO:tasks.workunit.client.0.trial172.stdout: "last_change": "26", 2026-02-24T18:09:45.058 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend": "0", 2026-02-24T18:09:45.058 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-24T18:09:45.059 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend_preluminous": "0", 2026-02-24T18:09:45.059 INFO:tasks.workunit.client.0.trial172.stdout: "auid": 0, 2026-02-24T18:09:45.059 INFO:tasks.workunit.client.0.trial172.stdout: "snap_mode": "selfmanaged", 2026-02-24T18:09:45.059 INFO:tasks.workunit.client.0.trial172.stdout: "snap_seq": 0, 2026-02-24T18:09:45.059 INFO:tasks.workunit.client.0.trial172.stdout: "snap_epoch": 0, 2026-02-24T18:09:45.059 INFO:tasks.workunit.client.0.trial172.stdout: "pool_snaps": [], 2026-02-24T18:09:45.059 INFO:tasks.workunit.client.0.trial172.stdout: "removed_snaps": "[]", 2026-02-24T18:09:45.059 INFO:tasks.workunit.client.0.trial172.stdout: "quota_max_bytes": 0, 2026-02-24T18:09:45.059 INFO:tasks.workunit.client.0.trial172.stdout: "quota_max_objects": 0, 2026-02-24T18:09:45.059 INFO:tasks.workunit.client.0.trial172.stdout: "tiers": [], 2026-02-24T18:09:45.059 INFO:tasks.workunit.client.0.trial172.stdout: "tier_of": -1, 2026-02-24T18:09:45.059 INFO:tasks.workunit.client.0.trial172.stdout: "read_tier": -1, 2026-02-24T18:09:45.059 INFO:tasks.workunit.client.0.trial172.stdout: "write_tier": -1, 2026-02-24T18:09:45.060 INFO:tasks.workunit.client.0.trial172.stdout: "cache_mode": "none", 2026-02-24T18:09:45.060 INFO:tasks.workunit.client.0.trial172.stdout: "target_max_bytes": 0, 2026-02-24T18:09:45.060 INFO:tasks.workunit.client.0.trial172.stdout: "target_max_objects": 0, 2026-02-24T18:09:45.060 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-24T18:09:45.060 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-24T18:09:45.060 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-24T18:09:45.060 INFO:tasks.workunit.client.0.trial172.stdout: "cache_min_flush_age": 0, 2026-02-24T18:09:45.060 INFO:tasks.workunit.client.0.trial172.stdout: "cache_min_evict_age": 0, 2026-02-24T18:09:45.060 INFO:tasks.workunit.client.0.trial172.stdout: "erasure_code_profile": "", 2026-02-24T18:09:45.060 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_params": { 2026-02-24T18:09:45.060 INFO:tasks.workunit.client.0.trial172.stdout: "type": "none" 2026-02-24T18:09:45.060 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.060 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_period": 0, 2026-02-24T18:09:45.061 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_count": 0, 2026-02-24T18:09:45.061 INFO:tasks.workunit.client.0.trial172.stdout: "use_gmt_hitset": true, 2026-02-24T18:09:45.061 INFO:tasks.workunit.client.0.trial172.stdout: "min_read_recency_for_promote": 0, 2026-02-24T18:09:45.061 INFO:tasks.workunit.client.0.trial172.stdout: "min_write_recency_for_promote": 0, 2026-02-24T18:09:45.061 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_grade_decay_rate": 0, 2026-02-24T18:09:45.061 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_search_last_n": 0, 2026-02-24T18:09:45.061 INFO:tasks.workunit.client.0.trial172.stdout: "grade_table": [], 2026-02-24T18:09:45.061 INFO:tasks.workunit.client.0.trial172.stdout: "stripe_width": 0, 2026-02-24T18:09:45.061 INFO:tasks.workunit.client.0.trial172.stdout: "expected_num_objects": 0, 2026-02-24T18:09:45.061 INFO:tasks.workunit.client.0.trial172.stdout: "fast_read": false, 2026-02-24T18:09:45.061 INFO:tasks.workunit.client.0.trial172.stdout: "nonprimary_shards": "{}", 2026-02-24T18:09:45.061 INFO:tasks.workunit.client.0.trial172.stdout: "options": {}, 2026-02-24T18:09:45.061 INFO:tasks.workunit.client.0.trial172.stdout: "application_metadata": {}, 2026-02-24T18:09:45.062 INFO:tasks.workunit.client.0.trial172.stdout: "read_balance": { 2026-02-24T18:09:45.062 INFO:tasks.workunit.client.0.trial172.stdout: "score_type": "Fair distribution", 2026-02-24T18:09:45.062 INFO:tasks.workunit.client.0.trial172.stdout: "score_acting": 2.9900000095367432, 2026-02-24T18:09:45.062 INFO:tasks.workunit.client.0.trial172.stdout: "score_stable": 2.9900000095367432, 2026-02-24T18:09:45.062 INFO:tasks.workunit.client.0.trial172.stdout: "optimal_score": 0.67000001668930054, 2026-02-24T18:09:45.062 INFO:tasks.workunit.client.0.trial172.stdout: "raw_score_acting": 2, 2026-02-24T18:09:45.062 INFO:tasks.workunit.client.0.trial172.stdout: "raw_score_stable": 2, 2026-02-24T18:09:45.062 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity_weighted": 1, 2026-02-24T18:09:45.062 INFO:tasks.workunit.client.0.trial172.stdout: "average_primary_affinity": 1, 2026-02-24T18:09:45.062 INFO:tasks.workunit.client.0.trial172.stdout: "average_primary_affinity_weighted": 1 2026-02-24T18:09:45.062 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.062 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.062 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.063 INFO:tasks.workunit.client.0.trial172.stdout: "osds": [ 2026-02-24T18:09:45.063 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.063 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:09:45.063 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "b38c0560-5794-43f5-b966-bdacdd9b4be4", 2026-02-24T18:09:45.063 INFO:tasks.workunit.client.0.trial172.stdout: "up": 1, 2026-02-24T18:09:45.063 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:09:45.063 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:09:45.063 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:09:45.063 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 5, 2026-02-24T18:09:45.063 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 26, 2026-02-24T18:09:45.063 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 33, 2026-02-24T18:09:45.063 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 37, 2026-02-24T18:09:45.063 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 27, 2026-02-24T18:09:45.063 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:09:45.064 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:09:45.064 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:09:45.064 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.064 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:09:45.064 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6800", 2026-02-24T18:09:45.064 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1337209777 2026-02-24T18:09:45.064 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.064 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.064 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:09:45.064 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6801", 2026-02-24T18:09:45.064 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1337209777 2026-02-24T18:09:45.064 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.065 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.065 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.065 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:09:45.065 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:09:45.065 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.065 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:09:45.065 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6802", 2026-02-24T18:09:45.065 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1337209777 2026-02-24T18:09:45.065 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.065 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.065 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:09:45.065 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6803", 2026-02-24T18:09:45.065 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1337209777 2026-02-24T18:09:45.066 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.066 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.066 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.066 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:09:45.066 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:09:45.066 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.066 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:09:45.066 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6806", 2026-02-24T18:09:45.066 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1337209777 2026-02-24T18:09:45.066 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.066 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.066 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:09:45.066 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6807", 2026-02-24T18:09:45.067 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1337209777 2026-02-24T18:09:45.067 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.067 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.067 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.067 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:09:45.067 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:09:45.067 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.067 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:09:45.067 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6804", 2026-02-24T18:09:45.067 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1337209777 2026-02-24T18:09:45.067 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.067 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.067 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:09:45.068 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6805", 2026-02-24T18:09:45.068 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1337209777 2026-02-24T18:09:45.068 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.068 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.068 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.068 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6801/1337209777", 2026-02-24T18:09:45.068 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6803/1337209777", 2026-02-24T18:09:45.068 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6807/1337209777", 2026-02-24T18:09:45.068 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6805/1337209777", 2026-02-24T18:09:45.068 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:09:45.068 INFO:tasks.workunit.client.0.trial172.stdout: "exists", 2026-02-24T18:09:45.068 INFO:tasks.workunit.client.0.trial172.stdout: "up" 2026-02-24T18:09:45.068 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.069 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.069 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.069 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:09:45.069 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "65c8d646-b403-4a57-bc5c-0e3cbc649eb7", 2026-02-24T18:09:45.069 INFO:tasks.workunit.client.0.trial172.stdout: "up": 0, 2026-02-24T18:09:45.069 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:09:45.069 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:09:45.069 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:09:45.069 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 0, 2026-02-24T18:09:45.069 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 0, 2026-02-24T18:09:45.069 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 9, 2026-02-24T18:09:45.069 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 20, 2026-02-24T18:09:45.069 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 28, 2026-02-24T18:09:45.069 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:09:45.070 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:09:45.070 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:09:45.070 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.070 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:09:45.070 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6810", 2026-02-24T18:09:45.070 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2955788417 2026-02-24T18:09:45.070 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.070 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.070 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:09:45.070 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6811", 2026-02-24T18:09:45.070 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2955788417 2026-02-24T18:09:45.070 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.070 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.071 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.071 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:09:45.071 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:09:45.071 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.071 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:09:45.071 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6812", 2026-02-24T18:09:45.071 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2955788417 2026-02-24T18:09:45.071 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.071 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.071 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:09:45.072 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6813", 2026-02-24T18:09:45.072 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2955788417 2026-02-24T18:09:45.072 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.072 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.072 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.072 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:09:45.072 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:09:45.072 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.072 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:09:45.072 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6816", 2026-02-24T18:09:45.072 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2955788417 2026-02-24T18:09:45.072 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.073 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.073 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:09:45.073 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6817", 2026-02-24T18:09:45.073 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2955788417 2026-02-24T18:09:45.073 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.073 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.073 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.073 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:09:45.073 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:09:45.073 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.073 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:09:45.073 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6814", 2026-02-24T18:09:45.073 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2955788417 2026-02-24T18:09:45.074 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.074 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.074 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:09:45.074 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6815", 2026-02-24T18:09:45.074 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2955788417 2026-02-24T18:09:45.074 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.074 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.074 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.074 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6811/2955788417", 2026-02-24T18:09:45.074 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6813/2955788417", 2026-02-24T18:09:45.074 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2955788417", 2026-02-24T18:09:45.074 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2955788417", 2026-02-24T18:09:45.074 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:09:45.075 INFO:tasks.workunit.client.0.trial172.stdout: "exists" 2026-02-24T18:09:45.075 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.075 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.075 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.075 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:09:45.075 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "2df2f4e3-421e-48b6-b431-4981b05cb657", 2026-02-24T18:09:45.075 INFO:tasks.workunit.client.0.trial172.stdout: "up": 1, 2026-02-24T18:09:45.075 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:09:45.075 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:09:45.075 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:09:45.075 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 13, 2026-02-24T18:09:45.075 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 26, 2026-02-24T18:09:45.076 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 35, 2026-02-24T18:09:45.076 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 0, 2026-02-24T18:09:45.076 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 29, 2026-02-24T18:09:45.076 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:09:45.076 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:09:45.076 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:09:45.076 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.076 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:09:45.076 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6810", 2026-02-24T18:09:45.076 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2060230136 2026-02-24T18:09:45.076 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.076 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.076 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:09:45.077 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6811", 2026-02-24T18:09:45.077 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2060230136 2026-02-24T18:09:45.077 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.077 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.077 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.077 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:09:45.077 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:09:45.077 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.077 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:09:45.077 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6812", 2026-02-24T18:09:45.077 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2060230136 2026-02-24T18:09:45.077 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.077 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.077 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:09:45.078 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6813", 2026-02-24T18:09:45.078 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2060230136 2026-02-24T18:09:45.078 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.078 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.078 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.078 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:09:45.078 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:09:45.078 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.078 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:09:45.078 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6816", 2026-02-24T18:09:45.078 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2060230136 2026-02-24T18:09:45.078 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.078 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.079 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:09:45.079 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6817", 2026-02-24T18:09:45.079 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2060230136 2026-02-24T18:09:45.079 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.079 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.079 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.079 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:09:45.079 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:09:45.079 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.079 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:09:45.079 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6814", 2026-02-24T18:09:45.079 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2060230136 2026-02-24T18:09:45.079 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.080 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.080 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:09:45.080 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6815", 2026-02-24T18:09:45.080 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2060230136 2026-02-24T18:09:45.080 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.080 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.080 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.080 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6811/2060230136", 2026-02-24T18:09:45.080 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6813/2060230136", 2026-02-24T18:09:45.080 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2060230136", 2026-02-24T18:09:45.080 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2060230136", 2026-02-24T18:09:45.080 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:09:45.080 INFO:tasks.workunit.client.0.trial172.stdout: "exists", 2026-02-24T18:09:45.081 INFO:tasks.workunit.client.0.trial172.stdout: "up" 2026-02-24T18:09:45.081 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.081 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.081 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.081 INFO:tasks.workunit.client.0.trial172.stdout: "osd_xinfo": [ 2026-02-24T18:09:45.081 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.081 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:09:45.081 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:09:10.073506+0000", 2026-02-24T18:09:45.081 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:09:45.081 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:09:45.081 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:09:45.081 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:09:45.081 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:08:35.608518+0000", 2026-02-24T18:09:45.082 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:09:45.082 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.082 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.082 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:09:45.082 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:09:10.353792+0000", 2026-02-24T18:09:45.082 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:09:45.082 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:09:45.082 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:09:45.082 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:09:45.082 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-24T18:09:45.082 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:09:45.082 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.082 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.082 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:09:45.082 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:09:10.642149+0000", 2026-02-24T18:09:45.083 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:09:45.083 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:09:45.083 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:09:45.083 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:09:45.083 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:08:42.950158+0000", 2026-02-24T18:09:45.083 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:09:45.083 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.083 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.083 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap": [], 2026-02-24T18:09:45.083 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap_items": [], 2026-02-24T18:09:45.083 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap_primaries": [], 2026-02-24T18:09:45.083 INFO:tasks.workunit.client.0.trial172.stdout: "pg_temp": [], 2026-02-24T18:09:45.083 INFO:tasks.workunit.client.0.trial172.stdout: "primary_temp": [], 2026-02-24T18:09:45.083 INFO:tasks.workunit.client.0.trial172.stdout: "blocklist": {}, 2026-02-24T18:09:45.084 INFO:tasks.workunit.client.0.trial172.stdout: "range_blocklist": {}, 2026-02-24T18:09:45.084 INFO:tasks.workunit.client.0.trial172.stdout: "erasure_code_profiles": { 2026-02-24T18:09:45.084 INFO:tasks.workunit.client.0.trial172.stdout: "default": { 2026-02-24T18:09:45.084 INFO:tasks.workunit.client.0.trial172.stdout: "k": "2", 2026-02-24T18:09:45.084 INFO:tasks.workunit.client.0.trial172.stdout: "m": "2", 2026-02-24T18:09:45.084 INFO:tasks.workunit.client.0.trial172.stdout: "plugin": "isa", 2026-02-24T18:09:45.084 INFO:tasks.workunit.client.0.trial172.stdout: "technique": "reed_sol_van" 2026-02-24T18:09:45.084 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.084 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.084 INFO:tasks.workunit.client.0.trial172.stdout: "removed_snaps_queue": [], 2026-02-24T18:09:45.084 INFO:tasks.workunit.client.0.trial172.stdout: "new_removed_snaps": [], 2026-02-24T18:09:45.084 INFO:tasks.workunit.client.0.trial172.stdout: "new_purged_snaps": [], 2026-02-24T18:09:45.084 INFO:tasks.workunit.client.0.trial172.stdout: "crush_node_flags": {}, 2026-02-24T18:09:45.084 INFO:tasks.workunit.client.0.trial172.stdout: "device_class_flags": {}, 2026-02-24T18:09:45.084 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode": { 2026-02-24T18:09:45.085 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode_enabled": false, 2026-02-24T18:09:45.085 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_bucket_count": 0, 2026-02-24T18:09:45.085 INFO:tasks.workunit.client.0.trial172.stdout: "degraded_stretch_mode": 0, 2026-02-24T18:09:45.085 INFO:tasks.workunit.client.0.trial172.stdout: "recovering_stretch_mode": 0, 2026-02-24T18:09:45.085 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode_bucket": 0 2026-02-24T18:09:45.085 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.085 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.085 INFO:tasks.workunit.client.0.trial172.stdout: "osd_metadata": [ 2026-02-24T18:09:45.085 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.085 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:09:45.085 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:09:45.085 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6802/1337209777,v1:127.0.0.1:6803/1337209777]", 2026-02-24T18:09:45.085 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:09:45.085 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:09:45.086 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:08:32.766512Z", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:09:45.087 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6800/1337209777,v1:127.0.0.1:6801/1337209777]", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/1337209777,v1:127.0.0.1:6807/1337209777]", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/1337209777,v1:127.0.0.1:6805/1337209777]", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/0", 2026-02-24T18:09:45.088 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6812/2955788417,v1:127.0.0.1:6813/2955788417]", 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:09:45.089 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:09:45.090 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:08:36.408240Z", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6810/2955788417,v1:127.0.0.1:6811/2955788417]", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2955788417,v1:127.0.0.1:6817/2955788417]", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2955788417,v1:127.0.0.1:6815/2955788417]", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:09:45.091 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/1", 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6812/2060230136,v1:127.0.0.1:6813/2060230136]", 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:09:45.092 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:09:45.093 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:09:45.093 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:09:45.093 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:09:45.093 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:09:45.093 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:09:45.093 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:09:45.093 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:09:45.093 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:09:45.093 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:09:45.093 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:09:45.093 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-24T18:09:45.093 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:09:45.093 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:08:40.489082Z", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:09:45.094 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6810/2060230136,v1:127.0.0.1:6811/2060230136]", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2060230136,v1:127.0.0.1:6817/2060230136]", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2060230136,v1:127.0.0.1:6815/2060230136]", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/2", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:09:45.095 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_clean_epochs": { 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: "min_last_epoch_clean": 21, 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_clean": { 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: "per_pool": [ 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": 1, 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: "floor": 21 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: "osd_epochs": [ 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 38 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:09:45.096 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 38 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_first_committed": 1, 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_last_committed": 40, 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: "crushmap": { 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: "devices": [ 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.0", 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.1", 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:09:45.097 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.2", 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: "types": [ 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 0, 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd" 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: "name": "host" 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 2, 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: "name": "chassis" 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 3, 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: "name": "rack" 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.098 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 4, 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: "name": "row" 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 5, 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: "name": "pdu" 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 6, 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: "name": "pod" 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 7, 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: "name": "room" 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 8, 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: "name": "datacenter" 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 9, 2026-02-24T18:09:45.099 INFO:tasks.workunit.client.0.trial172.stdout: "name": "zone" 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 10, 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: "name": "region" 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: "name": "root" 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: "buckets": [ 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: "id": -1, 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: "name": "default", 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "root", 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:09:45.100 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: "id": -3, 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: "id": -2, 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: "name": "default~ssd", 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "root", 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: "id": -4, 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.101 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "id": -3, 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "name": "trial172", 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "host", 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 1 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:09:45.102 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 2 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: "id": -4, 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: "name": "trial172~ssd", 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "host", 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 1 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.103 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 2 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: "rules": [ 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: "rule_id": 0, 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: "rule_name": "replicated_rule", 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: "type": 1, 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: "steps": [ 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: "op": "take", 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: "item": -1, 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: "item_name": "default" 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: "op": "choose_firstn", 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: "num": 0, 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: "type": "osd" 2026-02-24T18:09:45.104 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.105 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.105 INFO:tasks.workunit.client.0.trial172.stdout: "op": "emit" 2026-02-24T18:09:45.105 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.105 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:09:45.105 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.105 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.105 INFO:tasks.workunit.client.0.trial172.stdout: "tunables": { 2026-02-24T18:09:45.105 INFO:tasks.workunit.client.0.trial172.stdout: "choose_local_tries": 0, 2026-02-24T18:09:45.105 INFO:tasks.workunit.client.0.trial172.stdout: "choose_local_fallback_tries": 0, 2026-02-24T18:09:45.105 INFO:tasks.workunit.client.0.trial172.stdout: "choose_total_tries": 50, 2026-02-24T18:09:45.105 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_descend_once": 1, 2026-02-24T18:09:45.105 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_vary_r": 1, 2026-02-24T18:09:45.105 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_stable": 1, 2026-02-24T18:09:45.105 INFO:tasks.workunit.client.0.trial172.stdout: "msr_descents": 100, 2026-02-24T18:09:45.105 INFO:tasks.workunit.client.0.trial172.stderr:report 1393699867 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "msr_collision_tries": 100, 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "straw_calc_version": 1, 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "allowed_bucket_algs": 54, 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "profile": "jewel", 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "optimal_tunables": 1, 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "legacy_tunables": 0, 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "minimum_required_version": "jewel", 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables": 1, 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables2": 1, 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "has_v2_rules": 0, 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables3": 1, 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "has_v3_rules": 0, 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "has_v4_buckets": 1, 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables5": 1, 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "has_v5_rules": 0, 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "has_msr_rules": 0 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "choose_args": {} 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "fsmap": { 2026-02-24T18:09:45.106 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 1, 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "btime": "2026-02-24T18:08:31:790460+0000", 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "default_fscid": -1, 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "compat": { 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "compat": {}, 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "ro_compat": {}, 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "incompat": { 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "feature_1": "base v0.20", 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "feature_2": "client writeable ranges", 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "feature_3": "default file layouts on dirs", 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "feature_4": "dir inode in separate object", 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "feature_5": "mds uses versioned encoding", 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "feature_8": "no anchor table", 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "feature_9": "file layout v2", 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "feature_10": "snaprealm v2", 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "feature_11": "minor log segments", 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "feature_12": "quiesce subvolumes" 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "feature_flags": { 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "enable_multiple": true, 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "ever_enabled_multiple": true 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.107 INFO:tasks.workunit.client.0.trial172.stdout: "standbys": [], 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "filesystems": [] 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "mdsmap_first_committed": 1, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "mdsmap_last_committed": 1, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "auth": { 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "first_committed": 1, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "last_committed": 5, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "num_secrets": 3 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg": 1, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_active": 1, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_unknown": 0, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "num_osd": 3, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "pool_sum": { 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "stat_sum": { 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes": 51200, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects": 100, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_clones": 0, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_copies": 300, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing": 0, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_degraded": 100, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_misplaced": 0, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_unfound": 0, 2026-02-24T18:09:45.108 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_dirty": 100, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_whiteouts": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_read": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_read_kb": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_write": 100, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_write_kb": 100, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_scrub_errors": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_deep_scrub_errors": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_recovered": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_recovered": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_keys_recovered": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_omap": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_kb": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_kb": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_promote": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_high": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_low": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_some": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_full": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_pinned": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_legacy_snapsets": 0, 2026-02-24T18:09:45.109 INFO:tasks.workunit.client.0.trial172.stdout: "num_large_omap_objects": 0, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_manifest": 0, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_bytes": 0, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_keys": 0, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_repaired": 0 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "store_stats": { 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "total": 0, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "available": 0, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 0, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 0, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 0, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 0 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "log_size": 16, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "ondisk_log_size": 16, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "up": 2, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "acting": 2, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "num_store_stats": 0 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "osd_sum": { 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 0, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 0, 2026-02-24T18:09:45.110 INFO:tasks.workunit.client.0.trial172.stdout: "num_pgs": 3, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "num_osds": 3, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_osds": 3, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_omap_osds": 3, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "kb": 314572800, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used": 83852, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_data": 2072, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_omap": 27, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_meta": 81700, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "kb_avail": 314488948, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "statfs": { 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "total": 322122547200, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "available": 322036682752, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 2121728, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 421200, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 28003, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 83661469 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "hb_peers": [], 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "snap_trim_queue_len": 0, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "num_snap_trimming": 0, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "num_shards_repaired": 1, 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "op_queue_age_hist": { 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "histogram": [], 2026-02-24T18:09:45.111 INFO:tasks.workunit.client.0.trial172.stdout: "upper_bound": 1 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "perf_stat": { 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ms": 0, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ms": 0, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ns": 0, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ns": 0 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "alerts": [], 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "network_ping_times": [] 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "osd_sum_by_class": { 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "ssd": { 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 0, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 0, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "num_pgs": 3, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "num_osds": 3, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_osds": 3, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_omap_osds": 3, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "kb": 314572800, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used": 83852, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_data": 2072, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_omap": 27, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_meta": 81700, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "kb_avail": 314488948, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "statfs": { 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "total": 322122547200, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "available": 322036682752, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:09:45.112 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 2121728, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 421200, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 28003, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 83661469 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "hb_peers": [], 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "snap_trim_queue_len": 0, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "num_snap_trimming": 0, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "num_shards_repaired": 1, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "op_queue_age_hist": { 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "histogram": [], 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "upper_bound": 1 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "perf_stat": { 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ms": 0, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ms": 0, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ns": 0, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ns": 0 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "alerts": [], 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "network_ping_times": [] 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "pool_stats": [ 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": 1, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg": 1, 2026-02-24T18:09:45.113 INFO:tasks.workunit.client.0.trial172.stdout: "stat_sum": { 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes": 51200, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects": 100, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_clones": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_copies": 300, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_degraded": 100, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_misplaced": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_unfound": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_dirty": 100, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_whiteouts": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_read": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_read_kb": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_write": 100, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_write_kb": 100, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_scrub_errors": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_deep_scrub_errors": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_recovered": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_recovered": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_keys_recovered": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_omap": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_kb": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_kb": 0, 2026-02-24T18:09:45.114 INFO:tasks.workunit.client.0.trial172.stdout: "num_promote": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_high": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_low": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_some": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_full": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_pinned": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "num_legacy_snapsets": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "num_large_omap_objects": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_manifest": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_bytes": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_keys": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_repaired": 0 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "store_stats": { 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "total": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "available": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 1228800, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 153600, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 0, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 0 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "log_size": 16, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "ondisk_log_size": 16, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "up": 2, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "acting": 2, 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: "num_store_stats": 3 2026-02-24T18:09:45.115 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: "osd_stats": [ 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 141733920777 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 38654705677 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 150323855368 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_by_state": [ 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: "state": "active+undersized+degraded", 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: "num": 1 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.116 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "pool_pg_unavailable_map": [ 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": "1", 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "pgs": [] 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_by_osd": [ 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 1, 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 1, 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 1, 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 0, 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snaps": [ 2026-02-24T18:09:45.117 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "pool": 1, 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snaps": {} 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "servicemap": { 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 3, 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:09:40.253272+0000", 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "services": { 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "osd": { 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "daemons": { 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "summary": "", 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "0": { 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "start_epoch": 0, 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "start_stamp": "0.000000", 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "gid": 0, 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "metadata": {}, 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "task_status": {} 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "2": { 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "start_epoch": 0, 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "start_stamp": "0.000000", 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "gid": 0, 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "metadata": {}, 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: "task_status": {} 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.118 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:09:45.119 INFO:tasks.workunit.client.0.trial172.stdout: "mgrstat_first_committed": 1, 2026-02-24T18:09:45.119 INFO:tasks.workunit.client.0.trial172.stdout: "mgrstat_last_committed": 37, 2026-02-24T18:09:45.119 INFO:tasks.workunit.client.0.trial172.stdout: "logm_first_committed": 1, 2026-02-24T18:09:45.119 INFO:tasks.workunit.client.0.trial172.stdout: "logm_last_committed": 221, 2026-02-24T18:09:45.119 INFO:tasks.workunit.client.0.trial172.stdout: "paxos": { 2026-02-24T18:09:45.119 INFO:tasks.workunit.client.0.trial172.stdout: "first_committed": 1, 2026-02-24T18:09:45.119 INFO:tasks.workunit.client.0.trial172.stdout: "last_committed": 309, 2026-02-24T18:09:45.119 INFO:tasks.workunit.client.0.trial172.stdout: "last_pn": 0, 2026-02-24T18:09:45.119 INFO:tasks.workunit.client.0.trial172.stdout: "accepted_pn": 0 2026-02-24T18:09:45.119 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:09:45.119 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:09:45.119 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-24T18:09:45.119 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:150: TEST_divergent: expr 2 + 2 2026-02-24T18:09:45.119 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:150: TEST_divergent: objname=existing_4 2026-02-24T18:09:45.119 INFO:tasks.workunit.client.0.trial172.stdout:writing non-divergent object existing_4 2026-02-24T18:09:45.120 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:151: TEST_divergent: echo 'writing non-divergent object existing_4' 2026-02-24T18:09:45.120 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:152: TEST_divergent: ceph pg dump pgs 2026-02-24T18:09:45.266 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:09:45.266 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 100 0 0 51200 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:09:15.417142+0000 26'100 40:133 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:08:44.855101+0000 0'0 2026-02-24T18:08:44.855101+0000 0 0 periodic scrub scheduled @ 2026-02-26T02:22:05.234935+0000 0 0 2026-02-24T18:09:45.266 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:09:45.266 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:09:45.266 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:09:45.275 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:154: TEST_divergent: dd if=/dev/urandom bs=1000 count=1 2026-02-24T18:09:45.275 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:154: TEST_divergent: rados -p test put existing_4 - 2026-02-24T18:09:45.276 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records in 2026-02-24T18:09:45.276 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records out 2026-02-24T18:09:45.276 INFO:tasks.workunit.client.0.trial172.stderr:1000 bytes (1.0 kB) copied, 1.9426e-05 s, 51.5 MB/s 2026-02-24T18:09:45.298 INFO:tasks.workunit.client.0.trial172.stdout:delay recovery 2026-02-24T18:09:45.298 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:157: TEST_divergent: echo 'delay recovery' 2026-02-24T18:09:45.298 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:158: TEST_divergent: ceph pg dump pgs 2026-02-24T18:09:45.496 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:09:45.497 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 100 0 0 51200 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:09:15.417142+0000 26'100 40:133 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:08:44.855101+0000 0'0 2026-02-24T18:08:44.855101+0000 0 0 periodic scrub scheduled @ 2026-02-26T02:22:05.234935+0000 0 0 2026-02-24T18:09:45.497 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:09:45.497 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:09:45.497 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:09:45.507 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:159: TEST_divergent: for i in $non_divergent 2026-02-24T18:09:45.508 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: get_asok_path osd.0 2026-02-24T18:09:45.508 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T18:09:45.508 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T18:09:45.508 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:09:45.508 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:09:45.508 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:09:45.509 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.0.asok 2026-02-24T18:09:45.509 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: CEPH_ARGS= 2026-02-24T18:09:45.509 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.0.asok set_recovery_delay 100000 2026-02-24T18:09:45.575 INFO:tasks.workunit.client.0.trial172.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-24T18:09:45.581 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:159: TEST_divergent: for i in $non_divergent 2026-02-24T18:09:45.582 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: get_asok_path osd.2 2026-02-24T18:09:45.582 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-24T18:09:45.582 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-24T18:09:45.583 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:09:45.583 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:09:45.583 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:09:45.583 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.2.asok 2026-02-24T18:09:45.583 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: CEPH_ARGS= 2026-02-24T18:09:45.583 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.2.asok set_recovery_delay 100000 2026-02-24T18:09:45.639 INFO:tasks.workunit.client.0.trial172.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-24T18:09:45.645 INFO:tasks.workunit.client.0.trial172.stdout:revive divergent 1 2026-02-24T18:09:45.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:165: TEST_divergent: echo 'revive divergent 1' 2026-02-24T18:09:45.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:166: TEST_divergent: ceph pg dump pgs 2026-02-24T18:09:45.859 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:09:45.859 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 100 0 0 51200 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:09:15.417142+0000 26'100 40:133 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:08:44.855101+0000 0'0 2026-02-24T18:08:44.855101+0000 0 0 periodic scrub scheduled @ 2026-02-26T02:22:05.234935+0000 0 0 2026-02-24T18:09:45.860 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:09:45.860 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:09:45.860 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:09:45.870 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:167: TEST_divergent: ceph osd set noup 2026-02-24T18:09:46.181 INFO:tasks.workunit.client.0.trial172.stderr:noup is set 2026-02-24T18:09:46.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:168: TEST_divergent: activate_osd td/divergent-priors 1 2026-02-24T18:09:46.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:09:46.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:09:46.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T18:09:46.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:09:46.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-24T18:09:46.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:09:46.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:09:46.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:09:46.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:09:46.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-24T18:09:46.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-24T18:09:46.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:09:46.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:09:46.192 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:09:46.192 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:09:46.192 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:09:46.192 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:09:46.192 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:09:46.192 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:09:46.192 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:09:46.192 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:09:46.192 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:09:46.192 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:09:46.192 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:09:46.192 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:09:46.193 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:09:46.193 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:09:46.193 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:09:46.193 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:09:46.193 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:09:46.193 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:09:46.193 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-24T18:09:46.193 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:09:46.193 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T18:09:46.194 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:09:46.194 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-24T18:09:46.195 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T18:09:46.195 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:09:46.195 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:09:46.195 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:09:46.218 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:46.216+0000 7f3131bc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:09:46.220 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:46.216+0000 7f3131bc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:09:46.221 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:46.216+0000 7f3131bc1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:09:46.410 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: return 0 2026-02-24T18:09:46.410 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:169: TEST_divergent: sleep 5 2026-02-24T18:09:46.579 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:46.576+0000 7f3131bc1a40 -1 Falling back to public interface 2026-02-24T18:09:47.326 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:47.324+0000 7f3131bc1a40 -1 osd.1 26 log_to_monitors true 2026-02-24T18:09:51.412 INFO:tasks.workunit.client.0.trial172.stdout:delay recovery divergent 2026-02-24T18:09:51.412 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:171: TEST_divergent: echo 'delay recovery divergent' 2026-02-24T18:09:51.412 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:172: TEST_divergent: ceph pg dump pgs 2026-02-24T18:09:51.613 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:09:51.613 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 100 0 0 51688 0 0 17 84 17 active+undersized+degraded 2026-02-24T18:09:15.417142+0000 40'101 43:140 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:08:44.855101+0000 0'0 2026-02-24T18:08:44.855101+0000 0 0 periodic scrub scheduled @ 2026-02-26T02:22:05.234935+0000 0 0 2026-02-24T18:09:51.613 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:09:51.613 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:09:51.613 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:09:51.624 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:173: TEST_divergent: get_asok_path osd.1 2026-02-24T18:09:51.624 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T18:09:51.624 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T18:09:51.624 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:09:51.624 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:09:51.624 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:09:51.625 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.1.asok 2026-02-24T18:09:51.625 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:173: TEST_divergent: CEPH_ARGS= 2026-02-24T18:09:51.625 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:173: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.1.asok set_recovery_delay 100000 2026-02-24T18:09:51.692 INFO:tasks.workunit.client.0.trial172.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-24T18:09:51.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:175: TEST_divergent: ceph osd unset noup 2026-02-24T18:09:52.024 INFO:tasks.workunit.client.0.trial172.stderr:noup is unset 2026-02-24T18:09:52.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:177: TEST_divergent: wait_for_osd up 0 2026-02-24T18:09:52.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:09:52.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:09:52.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:09:52.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:09:52.035 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:09:52.036 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:09:52.036 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:09:52.036 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:09:52.036 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:09:52.257 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 33 up_thru 37 down_at 27 last_clean_interval [5,26) [v2:127.0.0.1:6800/1337209777,v1:127.0.0.1:6801/1337209777] [v2:127.0.0.1:6802/1337209777,v1:127.0.0.1:6803/1337209777] exists,up b38c0560-5794-43f5-b966-bdacdd9b4be4 2026-02-24T18:09:52.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:09:52.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:09:52.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:09:52.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:178: TEST_divergent: wait_for_osd up 1 2026-02-24T18:09:52.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:09:52.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:09:52.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:09:52.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:09:52.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:09:52.258 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:09:52.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:09:52.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:09:52.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:09:52.481 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 45 up_thru 45 down_at 28 last_clean_interval [9,26) [v2:127.0.0.1:6818/2534458819,v1:127.0.0.1:6819/2534458819] [v2:127.0.0.1:6820/2534458819,v1:127.0.0.1:6821/2534458819] exists,up 65c8d646-b403-4a57-bc5c-0e3cbc649eb7 2026-02-24T18:09:52.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:09:52.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:09:52.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:09:52.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:179: TEST_divergent: wait_for_osd up 2 2026-02-24T18:09:52.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:09:52.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:09:52.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:09:52.482 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:09:52.482 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:09:52.482 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:09:52.482 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:09:52.482 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:09:52.482 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:09:52.700 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 35 up_thru 0 down_at 29 last_clean_interval [13,26) [v2:127.0.0.1:6810/2060230136,v1:127.0.0.1:6811/2060230136] [v2:127.0.0.1:6812/2060230136,v1:127.0.0.1:6813/2060230136] exists,up 2df2f4e3-421e-48b6-b431-4981b05cb657 2026-02-24T18:09:52.701 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:09:52.701 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:09:52.701 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:09:52.701 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:181: TEST_divergent: ceph pg dump pgs 2026-02-24T18:09:52.905 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:09:52.906 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 2 2 0 0 51688 0 0 17 84 17 active+recovering+degraded 2026-02-24T18:09:52.131864+0000 40'101 46:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:08:44.855101+0000 0'0 2026-02-24T18:08:44.855101+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:09:52.906 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:09:52.906 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:09:52.906 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:09:52.914 INFO:tasks.workunit.client.0.trial172.stdout:wait for peering 2026-02-24T18:09:52.915 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:182: TEST_divergent: echo 'wait for peering' 2026-02-24T18:09:52.915 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:183: TEST_divergent: ceph pg dump pgs 2026-02-24T18:09:53.131 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:09:53.131 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 2 2 0 0 51688 0 0 17 84 17 active+recovering+degraded 2026-02-24T18:09:52.131864+0000 40'101 46:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:08:44.855101+0000 0'0 2026-02-24T18:08:44.855101+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:09:53.131 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:09:53.131 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:09:53.132 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:09:53.142 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:184: TEST_divergent: rados -p test put foo /tmp/tmp.DTFILqR5zu 2026-02-24T18:09:53.166 INFO:tasks.workunit.client.0.trial172.stdout:killing divergent 1 2026-02-24T18:09:53.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:186: TEST_divergent: echo 'killing divergent 1' 2026-02-24T18:09:53.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:187: TEST_divergent: ceph pg dump pgs 2026-02-24T18:09:53.373 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:09:53.373 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 2 2 0 0 51688 0 0 17 84 17 active+recovering+degraded 2026-02-24T18:09:52.131864+0000 40'101 46:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:08:44.855101+0000 0'0 2026-02-24T18:08:44.855101+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:09:53.373 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:09:53.373 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:09:53.373 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:09:53.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:188: TEST_divergent: kill_daemons td/divergent-priors KILL osd.1 2026-02-24T18:09:53.382 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:09:53.382 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:09:53.383 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:09:53.383 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:09:53.383 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:09:53.490 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:09:53.490 INFO:tasks.workunit.client.0.trial172.stdout:reviving divergent 1 2026-02-24T18:09:53.491 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:190: TEST_divergent: echo 'reviving divergent 1' 2026-02-24T18:09:53.491 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:191: TEST_divergent: ceph pg dump pgs 2026-02-24T18:09:53.693 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:09:53.693 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 2 2 0 0 51688 0 0 17 84 17 active+recovering+degraded 2026-02-24T18:09:52.131864+0000 40'101 46:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:08:44.855101+0000 0'0 2026-02-24T18:08:44.855101+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:09:53.693 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:09:53.693 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:09:53.694 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:09:53.705 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:192: TEST_divergent: activate_osd td/divergent-priors 1 2026-02-24T18:09:53.705 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:09:53.705 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:09:53.705 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T18:09:53.705 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:09:53.705 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-24T18:09:53.705 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:09:53.705 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:09:53.705 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:09:53.706 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:09:53.706 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-24T18:09:53.706 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-24T18:09:53.706 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:09:53.706 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:09:53.706 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:09:53.706 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:09:53.706 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:09:53.706 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:09:53.706 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:09:53.706 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:09:53.706 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:09:53.707 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:09:53.707 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:09:53.707 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:09:53.707 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:09:53.707 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:09:53.707 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:09:53.707 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:09:53.707 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:09:53.707 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:09:53.707 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:09:53.707 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:09:53.708 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-24T18:09:53.708 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:09:53.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T18:09:53.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:09:53.709 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-24T18:09:53.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T18:09:53.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:09:53.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:09:53.710 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:09:53.733 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:53.727+0000 7f3156bc8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:09:53.734 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:53.731+0000 7f3156bc8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:09:53.735 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:53.731+0000 7f3156bc8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:09:53.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T18:09:53.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:09:53.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:09:53.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:09:53.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:09:53.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:09:53.932 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:09:53.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:09:53.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:09:53.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:09:54.132 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:54.127+0000 7f3156bc8a40 -1 Falling back to public interface 2026-02-24T18:09:54.149 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 45 up_thru 45 down_at 28 last_clean_interval [9,26) [v2:127.0.0.1:6818/2534458819,v1:127.0.0.1:6819/2534458819] [v2:127.0.0.1:6820/2534458819,v1:127.0.0.1:6821/2534458819] exists,up 65c8d646-b403-4a57-bc5c-0e3cbc649eb7 2026-02-24T18:09:54.150 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:09:54.150 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:09:54.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:09:54.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:194: TEST_divergent: sleep 20 2026-02-24T18:09:54.762 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:09:54.759+0000 7f3156bc8a40 -1 osd.1 47 log_to_monitors true 2026-02-24T18:10:14.151 INFO:tasks.workunit.client.0.trial172.stdout:allowing recovery 2026-02-24T18:10:14.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:196: TEST_divergent: echo 'allowing recovery' 2026-02-24T18:10:14.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:197: TEST_divergent: ceph pg dump pgs 2026-02-24T18:10:14.350 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:10:14.350 INFO:tasks.workunit.client.0.trial172.stdout:1.0 101 0 0 0 0 52200 0 0 18 84 18 active+clean 2026-02-24T18:09:55.190401+0000 47'102 51:161 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:08:44.855101+0000 0'0 2026-02-24T18:08:44.855101+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:10:16.834983+0000 0 0 2026-02-24T18:10:14.350 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:10:14.351 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:10:14.351 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:10:14.359 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:199: TEST_divergent: for i in $osds 2026-02-24T18:10:14.359 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-24T18:10:14.443 INFO:tasks.workunit.client.0.trial172.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-24T18:10:14.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:199: TEST_divergent: for i in $osds 2026-02-24T18:10:14.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-24T18:10:14.531 INFO:tasks.workunit.client.0.trial172.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-24T18:10:14.538 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:199: TEST_divergent: for i in $osds 2026-02-24T18:10:14.538 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-24T18:10:14.606 INFO:tasks.workunit.client.0.trial172.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-24T18:10:14.613 INFO:tasks.workunit.client.0.trial172.stdout:reading divergent objects 2026-02-24T18:10:14.614 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:204: TEST_divergent: echo 'reading divergent objects' 2026-02-24T18:10:14.614 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:205: TEST_divergent: ceph pg dump pgs 2026-02-24T18:10:14.827 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:10:14.828 INFO:tasks.workunit.client.0.trial172.stdout:1.0 101 0 0 0 0 52200 0 0 18 84 18 active+clean 2026-02-24T18:09:55.190401+0000 47'102 51:161 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:08:44.855101+0000 0'0 2026-02-24T18:08:44.855101+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:10:16.834983+0000 0 0 2026-02-24T18:10:14.828 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:10:14.828 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:10:14.828 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:10:14.837 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: expr 2 + 2 2026-02-24T18:10:14.838 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: seq 1 4 2026-02-24T18:10:14.839 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:10:14.839 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: rados -p test get existing_1 td/divergent-priors/existing 2026-02-24T18:10:14.861 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:10:14.861 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: rados -p test get existing_2 td/divergent-priors/existing 2026-02-24T18:10:14.889 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:10:14.889 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: rados -p test get existing_3 td/divergent-priors/existing 2026-02-24T18:10:14.916 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:10:14.916 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: rados -p test get existing_4 td/divergent-priors/existing 2026-02-24T18:10:14.943 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rm -f td/divergent-priors/existing 2026-02-24T18:10:14.945 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:212: TEST_divergent: find td/divergent-priors -name '*osd*log' 2026-02-24T18:10:14.946 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:212: TEST_divergent: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-24T18:10:14.951 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:09:52.127+0000 7f3113b47640 20 _merge_object_divergent_entries: merging hoid 1:c156033e:::existing_100:head entries: 26'101 (26'100) modify 1:c156033e:::existing_100:head by client.4534.0:1 2026-02-24T18:08:59.154740+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: true, new_object: false 2026-02-24T18:10:14.951 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:09:52.127+0000 7f3113b47640 20 _merge_object_divergent_entries: keeping 26'101 (26'100) modify 1:c156033e:::existing_100:head by client.4534.0:1 2026-02-24T18:08:59.154740+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: true, new_object: false 2026-02-24T18:10:14.951 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:09:52.127+0000 7f3113b47640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-24T18:10:14.951 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:09:52.127+0000 7f3113b47640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head prior_version: 26'100 first_divergent_update: 26'101 last_divergent_update: 26'101 2026-02-24T18:10:14.951 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:09:52.127+0000 7f3113b47640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head has no more recent entries in log 2026-02-24T18:10:14.951 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:09:52.127+0000 7f3113b47640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head must be rolled back or recovered, attempting to rollback 2026-02-24T18:10:14.951 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:09:52.127+0000 7f3113b47640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head olog_can_rollback_to: 26'101 2026-02-24T18:10:14.951 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:09:52.127+0000 7f3113b47640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot rollback 26'101 (26'100) modify 1:c156033e:::existing_100:head by client.4534.0:1 2026-02-24T18:08:59.154740+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: true, new_object: false 2026-02-24T18:10:14.951 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:09:52.127+0000 7f3113b47640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot roll back, removing and adding to missing 2026-02-24T18:10:14.951 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:214: TEST_divergent: find td/divergent-priors -name '*osd*log' 2026-02-24T18:10:14.952 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:214: TEST_divergent: grep -q '_merge_object_divergent_entries.*cannot roll back, removing and adding to missing' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-24T18:10:14.955 INFO:tasks.workunit.client.0.trial172.stdout:success 2026-02-24T18:10:14.955 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:219: TEST_divergent: echo success 2026-02-24T18:10:14.956 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:221: TEST_divergent: rm -f /tmp/tmp.DTFILqR5zu 2026-02-24T18:10:14.956 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:222: TEST_divergent: delete_pool test 2026-02-24T18:10:14.956 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=test 2026-02-24T18:10:14.956 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-24T18:10:15.215 INFO:tasks.workunit.client.0.trial172.stderr:pool 'test' does not exist 2026-02-24T18:10:15.226 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:223: TEST_divergent: kill_daemons td/divergent-priors 2026-02-24T18:10:15.227 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:10:15.228 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:10:15.228 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:10:15.228 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:10:15.228 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:10:20.543 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:10:20.543 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-24T18:10:20.543 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-24T18:10:20.543 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:10:20.543 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-24T18:10:20.544 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:10:20.544 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:10:20.544 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:10:20.544 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:10:20.544 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:10:20.550 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:10:20.551 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:10:20.551 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:10:20.552 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:10:20.553 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:10:20.553 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:10:20.554 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:10:20.555 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:10:20.555 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:10:20.555 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:10:20.555 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:10:20.556 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:10:20.557 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:10:20.557 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-24T18:10:20.577 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:10:20.577 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:10:20.577 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:10:20.577 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19163 2026-02-24T18:10:20.578 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:10:20.578 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:10:20.578 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-24T18:10:20.578 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-24T18:10:20.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/divergent-priors 2026-02-24T18:10:20.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/divergent-priors 2026-02-24T18:10:20.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-24T18:10:20.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:10:20.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-24T18:10:20.579 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:10:20.579 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:10:20.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:10:20.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:10:20.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:10:20.581 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:10:20.582 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:10:20.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:10:20.583 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:10:20.584 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:10:20.585 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:10:20.585 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:10:20.586 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:10:20.586 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:10:20.586 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:10:20.587 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:10:20.587 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:10:20.588 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:10:20.589 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-24T18:10:20.590 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:10:20.590 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:10:20.591 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:10:20.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19163 2026-02-24T18:10:20.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:10:20.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:10:20.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/divergent-priors 2026-02-24T18:10:20.594 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T18:10:20.594 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:10:20.594 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:10:20.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19163 2026-02-24T18:10:20.596 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T18:10:20.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T18:10:20.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T18:10:20.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-24T18:10:20.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_2 td/divergent-priors 2026-02-24T18:10:20.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:422: TEST_divergent_2: local dir=td/divergent-priors 2026-02-24T18:10:20.597 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:424: TEST_divergent_2: file_with_random_data 2026-02-24T18:10:20.597 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: file_with_random_data: local size_bytes=512 2026-02-24T18:10:20.598 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: file_with_random_data: mktemp 2026-02-24T18:10:20.600 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: file_with_random_data: local file=/tmp/tmp.AohTN555hP 2026-02-24T18:10:20.600 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2374: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.AohTN555hP bs=512 count=1 2026-02-24T18:10:20.600 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records in 2026-02-24T18:10:20.600 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records out 2026-02-24T18:10:20.600 INFO:tasks.workunit.client.0.trial172.stderr:512 bytes copied, 8.5689e-05 s, 6.0 MB/s 2026-02-24T18:10:20.600 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: file_with_random_data: printf %s /tmp/tmp.AohTN555hP 2026-02-24T18:10:20.601 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:424: TEST_divergent_2: local dummyfile=/tmp/tmp.AohTN555hP 2026-02-24T18:10:20.601 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:426: TEST_divergent_2: file_with_random_data 1000 2026-02-24T18:10:20.601 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: file_with_random_data: local size_bytes=1000 2026-02-24T18:10:20.601 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: file_with_random_data: mktemp 2026-02-24T18:10:20.603 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: file_with_random_data: local file=/tmp/tmp.OgRbPGtOtd 2026-02-24T18:10:20.603 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2374: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.OgRbPGtOtd bs=1000 count=1 2026-02-24T18:10:20.603 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records in 2026-02-24T18:10:20.603 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records out 2026-02-24T18:10:20.604 INFO:tasks.workunit.client.0.trial172.stderr:1000 bytes (1.0 kB) copied, 8.3665e-05 s, 12.0 MB/s 2026-02-24T18:10:20.604 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: file_with_random_data: printf %s /tmp/tmp.OgRbPGtOtd 2026-02-24T18:10:20.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:426: TEST_divergent_2: local dummyfile2=/tmp/tmp.OgRbPGtOtd 2026-02-24T18:10:20.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:428: TEST_divergent_2: local num_osds=3 2026-02-24T18:10:20.604 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:429: TEST_divergent_2: expr 3 - 1 2026-02-24T18:10:20.606 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:429: TEST_divergent_2: seq 0 2 2026-02-24T18:10:20.606 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:429: TEST_divergent_2: local 'osds=0 2026-02-24T18:10:20.606 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:10:20.606 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:10:20.606 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:430: TEST_divergent_2: run_mon td/divergent-priors a 2026-02-24T18:10:20.607 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/divergent-priors 2026-02-24T18:10:20.607 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T18:10:20.607 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T18:10:20.607 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T18:10:20.607 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/divergent-priors/a 2026-02-24T18:10:20.607 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-24T18:10:20.637 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T18:10:20.637 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:10:20.637 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:10:20.637 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:10:20.637 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:10:20.637 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:10:20.637 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:10:20.637 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T18:10:20.670 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T18:10:20.671 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T18:10:20.671 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:10:20.671 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:10:20.671 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T18:10:20.671 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T18:10:20.672 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:10:20.672 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:10:20.672 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:10:20.672 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:10:20.672 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:10:20.672 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:10:20.672 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-mon.a.asok 2026-02-24T18:10:20.673 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:10:20.673 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19163/ceph-mon.a.asok config get fsid 2026-02-24T18:10:20.730 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T18:10:20.730 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:10:20.730 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:10:20.730 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T18:10:20.730 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T18:10:20.731 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:10:20.731 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:10:20.731 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:10:20.731 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:10:20.731 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:10:20.731 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:10:20.731 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-mon.a.asok 2026-02-24T18:10:20.732 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:10:20.732 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19163/ceph-mon.a.asok config get mon_host 2026-02-24T18:10:20.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:431: TEST_divergent_2: run_mgr td/divergent-priors x 2026-02-24T18:10:20.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/divergent-priors 2026-02-24T18:10:20.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T18:10:20.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T18:10:20.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T18:10:20.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/divergent-priors/x 2026-02-24T18:10:20.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T18:10:20.907 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T18:10:20.907 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:10:20.907 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:10:20.907 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:10:20.907 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:10:20.907 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:10:20.908 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:10:20.908 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:10:20.909 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:10:20.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:432: TEST_divergent_2: for i in $osds 2026-02-24T18:10:20.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:434: TEST_divergent_2: run_osd td/divergent-priors 0 2026-02-24T18:10:20.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-24T18:10:20.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:10:20.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T18:10:20.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:10:20.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/0 2026-02-24T18:10:20.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:10:20.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:10:20.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:10:20.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:10:20.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-24T18:10:20.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-24T18:10:20.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:10:20.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:10:20.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:10:20.931 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:10:20.931 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:10:20.931 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:10:20.931 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:10:20.931 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:10:20.932 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:10:20.932 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:10:20.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:10:20.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:10:20.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:10:20.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:10:20.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:10:20.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:10:20.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:10:20.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:10:20.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:10:20.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:10:20.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:10:20.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:10:20.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/0 2026-02-24T18:10:20.933 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:10:20.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0d39d29d-6e3b-4fa3-a91e-35af674e24c9 2026-02-24T18:10:20.934 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0d39d29d-6e3b-4fa3-a91e-35af674e24c9' 2026-02-24T18:10:20.934 INFO:tasks.workunit.client.0.trial172.stdout:add osd0 0d39d29d-6e3b-4fa3-a91e-35af674e24c9 2026-02-24T18:10:20.934 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:10:20.945 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCM6Z1pVrA+OBAAf/yF3jC8nxQHIStBe2SZrg== 2026-02-24T18:10:20.945 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCM6Z1pVrA+OBAAf/yF3jC8nxQHIStBe2SZrg=="}' 2026-02-24T18:10:20.945 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0d39d29d-6e3b-4fa3-a91e-35af674e24c9 -i td/divergent-priors/0/new.json 2026-02-24T18:10:21.060 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:10:21.068 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/0/new.json 2026-02-24T18:10:21.069 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCM6Z1pVrA+OBAAf/yF3jC8nxQHIStBe2SZrg== --osd-uuid 0d39d29d-6e3b-4fa3-a91e-35af674e24c9 2026-02-24T18:10:21.087 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:21.083+0000 7fa851872a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:21.089 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:21.083+0000 7fa851872a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:21.090 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:21.087+0000 7fa851872a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:21.090 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:21.087+0000 7fa851872a40 -1 bdev(0x55ac9860e400 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-24T18:10:21.091 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:21.087+0000 7fa851872a40 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-24T18:10:21.878 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-24T18:10:21.878 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:10:21.879 INFO:tasks.workunit.client.0.trial172.stdout:adding osd0 key to auth repository 2026-02-24T18:10:21.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T18:10:21.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:10:22.007 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:10:22.008 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T18:10:22.009 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:10:22.009 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:10:22.009 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:10:22.009 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:10:22.038 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:22.031+0000 7fe9c0807a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:22.040 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:22.035+0000 7fe9c0807a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:22.041 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:22.035+0000 7fe9c0807a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:22.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T18:10:22.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:10:22.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:10:22.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:10:22.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:10:22.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:10:22.124 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:10:22.125 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:10:22.125 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:10:22.125 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:10:22.239 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:10:22.543 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:22.539+0000 7fe9c0807a40 -1 Falling back to public interface 2026-02-24T18:10:23.240 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:10:23.241 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:10:23.241 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:10:23.241 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:10:23.241 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:10:23.241 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:10:23.464 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:23.459+0000 7fe9c0807a40 -1 osd.0 0 log_to_monitors true 2026-02-24T18:10:23.467 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:10:24.469 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:10:24.469 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:10:24.469 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:10:24.470 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:10:24.470 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:10:24.470 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:10:24.689 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2943175813,v1:127.0.0.1:6801/2943175813] [v2:127.0.0.1:6802/2943175813,v1:127.0.0.1:6803/2943175813] exists,up 0d39d29d-6e3b-4fa3-a91e-35af674e24c9 2026-02-24T18:10:24.695 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:10:24.695 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:10:24.695 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:10:24.695 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:432: TEST_divergent_2: for i in $osds 2026-02-24T18:10:24.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:434: TEST_divergent_2: run_osd td/divergent-priors 1 2026-02-24T18:10:24.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-24T18:10:24.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:10:24.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T18:10:24.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:10:24.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/1 2026-02-24T18:10:24.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:10:24.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:10:24.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:10:24.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:10:24.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-24T18:10:24.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-24T18:10:24.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:10:24.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:10:24.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:10:24.697 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:10:24.697 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:10:24.697 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:10:24.698 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:10:24.698 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:10:24.698 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:10:24.698 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:10:24.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:10:24.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:10:24.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:10:24.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:10:24.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:10:24.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:10:24.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:10:24.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:10:24.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:10:24.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:10:24.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:10:24.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:10:24.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/1 2026-02-24T18:10:24.699 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:10:24.700 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d7cb9f1a-c825-4426-aded-3118e4460724 2026-02-24T18:10:24.700 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 d7cb9f1a-c825-4426-aded-3118e4460724' 2026-02-24T18:10:24.700 INFO:tasks.workunit.client.0.trial172.stdout:add osd1 d7cb9f1a-c825-4426-aded-3118e4460724 2026-02-24T18:10:24.700 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:10:24.708 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCQ6Z1pVskYKhAAApHgAdOB0ckCQkpLbxjaWA== 2026-02-24T18:10:24.708 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCQ6Z1pVskYKhAAApHgAdOB0ckCQkpLbxjaWA=="}' 2026-02-24T18:10:24.708 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d7cb9f1a-c825-4426-aded-3118e4460724 -i td/divergent-priors/1/new.json 2026-02-24T18:10:24.937 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:10:24.946 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/1/new.json 2026-02-24T18:10:24.947 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCQ6Z1pVskYKhAAApHgAdOB0ckCQkpLbxjaWA== --osd-uuid d7cb9f1a-c825-4426-aded-3118e4460724 2026-02-24T18:10:24.974 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:24.971+0000 7f7dc5018a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:24.976 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:24.971+0000 7f7dc5018a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:24.977 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:24.971+0000 7f7dc5018a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:24.977 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:24.971+0000 7f7dc5018a40 -1 bdev(0x557bdaea0400 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-24T18:10:24.978 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:24.971+0000 7f7dc5018a40 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-24T18:10:25.669 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-24T18:10:25.669 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:10:25.670 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T18:10:25.670 INFO:tasks.workunit.client.0.trial172.stdout:adding osd1 key to auth repository 2026-02-24T18:10:25.670 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:10:25.967 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:10:25.968 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T18:10:25.968 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:10:25.968 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:10:25.968 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:10:25.968 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:10:25.996 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:25.991+0000 7f060360ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:25.997 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:25.991+0000 7f060360ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:25.998 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:25.995+0000 7f060360ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:26.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T18:10:26.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:10:26.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:10:26.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:10:26.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:10:26.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:10:26.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:10:26.185 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:10:26.186 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:10:26.186 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:10:26.392 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:26.386+0000 7f060360ca40 -1 Falling back to public interface 2026-02-24T18:10:26.399 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:10:27.016 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:27.010+0000 7f060360ca40 -1 osd.1 0 log_to_monitors true 2026-02-24T18:10:27.402 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:10:27.402 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:10:27.402 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:10:27.403 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:10:27.403 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:10:27.403 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:10:27.627 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3631029146,v1:127.0.0.1:6811/3631029146] [v2:127.0.0.1:6812/3631029146,v1:127.0.0.1:6813/3631029146] exists,up d7cb9f1a-c825-4426-aded-3118e4460724 2026-02-24T18:10:27.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:10:27.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:10:27.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:10:27.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:432: TEST_divergent_2: for i in $osds 2026-02-24T18:10:27.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:434: TEST_divergent_2: run_osd td/divergent-priors 2 2026-02-24T18:10:27.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-24T18:10:27.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:10:27.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T18:10:27.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:10:27.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/2 2026-02-24T18:10:27.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:10:27.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:10:27.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:10:27.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:10:27.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-24T18:10:27.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-24T18:10:27.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:10:27.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:10:27.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:10:27.632 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:10:27.633 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:10:27.633 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:10:27.633 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:10:27.633 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:10:27.633 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:10:27.633 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:10:27.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:10:27.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:10:27.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:10:27.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:10:27.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:10:27.634 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:10:27.634 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:10:27.634 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:10:27.634 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:10:27.634 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:10:27.634 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:10:27.634 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:10:27.634 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/2 2026-02-24T18:10:27.634 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:10:27.635 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=aae44b3b-9b8d-43f3-925b-7313233d9556 2026-02-24T18:10:27.635 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 aae44b3b-9b8d-43f3-925b-7313233d9556' 2026-02-24T18:10:27.635 INFO:tasks.workunit.client.0.trial172.stdout:add osd2 aae44b3b-9b8d-43f3-925b-7313233d9556 2026-02-24T18:10:27.636 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:10:27.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCT6Z1p5WVkJhAA5ieDs0JAlUFUPVi0dP9Qbg== 2026-02-24T18:10:27.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCT6Z1p5WVkJhAA5ieDs0JAlUFUPVi0dP9Qbg=="}' 2026-02-24T18:10:27.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new aae44b3b-9b8d-43f3-925b-7313233d9556 -i td/divergent-priors/2/new.json 2026-02-24T18:10:27.864 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:10:27.874 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/2/new.json 2026-02-24T18:10:27.875 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCT6Z1p5WVkJhAA5ieDs0JAlUFUPVi0dP9Qbg== --osd-uuid aae44b3b-9b8d-43f3-925b-7313233d9556 2026-02-24T18:10:27.894 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:27.886+0000 7fc1f0546a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:27.896 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:27.890+0000 7fc1f0546a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:27.897 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:27.890+0000 7fc1f0546a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:27.897 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:27.890+0000 7fc1f0546a40 -1 bdev(0x55c81caea400 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-24T18:10:27.898 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:27.890+0000 7fc1f0546a40 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-24T18:10:28.538 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-24T18:10:28.538 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:10:28.539 INFO:tasks.workunit.client.0.trial172.stdout:adding osd2 key to auth repository 2026-02-24T18:10:28.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T18:10:28.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:10:28.844 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:10:28.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T18:10:28.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:10:28.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:10:28.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:10:28.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:10:28.864 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:28.857+0000 7fa528ff7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:28.865 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:28.857+0000 7fa528ff7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:28.866 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:28.857+0000 7fa528ff7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:29.059 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T18:10:29.059 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:10:29.059 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:10:29.059 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:10:29.059 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:10:29.059 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:10:29.060 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:10:29.060 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:10:29.060 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:10:29.061 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:10:29.273 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:29.265+0000 7fa528ff7a40 -1 Falling back to public interface 2026-02-24T18:10:29.282 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:10:29.917 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:29.909+0000 7fa528ff7a40 -1 osd.2 0 log_to_monitors true 2026-02-24T18:10:30.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:10:30.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:10:30.285 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:10:30.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:10:30.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:10:30.286 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:10:30.511 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:10:31.513 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:10:31.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:10:31.514 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:10:31.515 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:10:31.515 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:10:31.515 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:10:31.732 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/345723987,v1:127.0.0.1:6819/345723987] [v2:127.0.0.1:6820/345723987,v1:127.0.0.1:6821/345723987] exists,up aae44b3b-9b8d-43f3-925b-7313233d9556 2026-02-24T18:10:31.732 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:10:31.732 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:10:31.732 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:10:31.732 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:437: TEST_divergent_2: ceph osd set noout 2026-02-24T18:10:32.023 INFO:tasks.workunit.client.0.trial172.stderr:noout is set 2026-02-24T18:10:32.034 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:438: TEST_divergent_2: ceph osd set noin 2026-02-24T18:10:32.327 INFO:tasks.workunit.client.0.trial172.stderr:noin is set 2026-02-24T18:10:32.338 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:439: TEST_divergent_2: ceph osd set nodown 2026-02-24T18:10:32.632 INFO:tasks.workunit.client.0.trial172.stderr:nodown is set 2026-02-24T18:10:32.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:440: TEST_divergent_2: create_pool test 1 1 2026-02-24T18:10:32.646 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-24T18:10:32.889 INFO:tasks.workunit.client.0.trial172.stderr:pool 'test' already exists 2026-02-24T18:10:32.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T18:10:33.902 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:441: TEST_divergent_2: ceph osd pool set test size 3 2026-02-24T18:10:34.227 INFO:tasks.workunit.client.0.trial172.stderr:set pool 1 size to 3 2026-02-24T18:10:34.238 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:442: TEST_divergent_2: ceph osd pool set test min_size 2 2026-02-24T18:10:34.530 INFO:tasks.workunit.client.0.trial172.stderr:set pool 1 min_size to 2 2026-02-24T18:10:34.542 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:444: TEST_divergent_2: flush_pg_stats 2026-02-24T18:10:34.542 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:10:34.542 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:10:34.768 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:10:34.769 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:10:34.769 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:10:34.769 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:10:34.769 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:10:34.769 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:10:34.851 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836485 2026-02-24T18:10:34.851 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836485 2026-02-24T18:10:34.852 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836485' 2026-02-24T18:10:34.852 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:10:34.852 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:10:34.934 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672964 2026-02-24T18:10:34.934 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672964 2026-02-24T18:10:34.934 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-24T18:10:34.934 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:10:34.935 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:10:35.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=60129542147 2026-02-24T18:10:35.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 60129542147 2026-02-24T18:10:35.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-60129542147' 2026-02-24T18:10:35.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:10:35.019 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836485 2026-02-24T18:10:35.019 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:10:35.020 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:10:35.021 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836485 2026-02-24T18:10:35.021 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:10:35.022 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836485 2026-02-24T18:10:35.022 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 21474836485 2026-02-24T18:10:35.023 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-24T18:10:35.023 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:10:35.245 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-24T18:10:35.245 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:10:35.246 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672964 2026-02-24T18:10:35.246 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:10:35.247 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:10:35.247 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672964 2026-02-24T18:10:35.248 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:10:35.249 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672964 2026-02-24T18:10:35.249 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-24T18:10:35.249 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 42949672964 2026-02-24T18:10:35.249 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:10:35.466 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-24T18:10:35.466 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:10:35.467 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-60129542147 2026-02-24T18:10:35.467 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:10:35.468 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:10:35.469 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-60129542147 2026-02-24T18:10:35.469 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:10:35.470 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=60129542147 2026-02-24T18:10:35.470 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-24T18:10:35.471 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 60129542147 2026-02-24T18:10:35.471 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:10:35.689 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-24T18:10:35.689 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:445: TEST_divergent_2: wait_for_clean 2026-02-24T18:10:35.689 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:10:35.689 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:10:35.689 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:10:35.690 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T18:10:35.690 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:10:35.690 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:10:35.690 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:10:35.690 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:10:35.691 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:10:35.789 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T18:10:35.789 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:10:35.790 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:10:35.790 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:10:35.790 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:10:35.790 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:10:36.010 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:10:36.010 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:10:36.010 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:10:36.010 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:10:36.011 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:10:36.011 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:10:36.091 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836487 2026-02-24T18:10:36.091 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836487 2026-02-24T18:10:36.092 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836487' 2026-02-24T18:10:36.092 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:10:36.092 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:10:36.175 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672966 2026-02-24T18:10:36.175 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672966 2026-02-24T18:10:36.176 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836487 1-42949672966' 2026-02-24T18:10:36.176 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:10:36.176 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:10:36.255 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=60129542150 2026-02-24T18:10:36.255 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 60129542150 2026-02-24T18:10:36.255 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836487 1-42949672966 2-60129542150' 2026-02-24T18:10:36.255 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:10:36.256 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836487 2026-02-24T18:10:36.256 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:10:36.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:10:36.257 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836487 2026-02-24T18:10:36.257 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:10:36.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836487 2026-02-24T18:10:36.259 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 21474836487 2026-02-24T18:10:36.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-24T18:10:36.259 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:10:36.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-24T18:10:36.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:10:37.486 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:10:37.486 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:10:37.702 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-24T18:10:37.702 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:10:37.703 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672966 2026-02-24T18:10:37.703 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:10:37.703 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:10:37.704 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672966 2026-02-24T18:10:37.704 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:10:37.705 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672966 2026-02-24T18:10:37.706 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 42949672966 2026-02-24T18:10:37.706 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-24T18:10:37.706 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:10:37.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672966 -lt 42949672966 2026-02-24T18:10:37.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:10:37.941 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-60129542150 2026-02-24T18:10:37.941 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:10:37.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:10:37.943 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-60129542150 2026-02-24T18:10:37.943 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:10:37.943 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=60129542150 2026-02-24T18:10:37.944 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 60129542150 2026-02-24T18:10:37.945 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-24T18:10:37.945 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:10:38.176 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-24T18:10:38.176 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:10:38.177 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:10:38.178 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:10:38.463 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:10:38.463 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:10:38.463 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:10:38.463 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:10:38.463 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:10:38.463 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:10:38.464 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:10:38.464 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:10:38.676 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-24T18:10:38.677 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:10:38.677 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:10:38.677 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:10:38.960 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-24T18:10:38.960 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T18:10:38.960 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T18:10:38.960 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:448: TEST_divergent_2: ceph pg dump pgs --format=json 2026-02-24T18:10:38.961 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:448: TEST_divergent_2: jq '.pg_stats[0].up_primary' 2026-02-24T18:10:39.157 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:10:39.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:448: TEST_divergent_2: local divergent=1 2026-02-24T18:10:39.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:449: TEST_divergent_2: echo 'primary and soon to be divergent is 1' 2026-02-24T18:10:39.167 INFO:tasks.workunit.client.0.trial172.stdout:primary and soon to be divergent is 1 2026-02-24T18:10:39.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:450: TEST_divergent_2: ceph pg dump pgs 2026-02-24T18:10:39.371 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:10:39.371 INFO:tasks.workunit.client.0.trial172.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T18:10:33.767543+0000 0'0 27:22 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:10:32.829947+0000 0'0 2026-02-24T18:10:32.829947+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:56:53.688698+0000 0 0 2026-02-24T18:10:39.371 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:10:39.371 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:10:39.371 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:10:39.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:451: TEST_divergent_2: local non_divergent= 2026-02-24T18:10:39.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:452: TEST_divergent_2: for i in $osds 2026-02-24T18:10:39.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:454: TEST_divergent_2: '[' 0 = 1 ']' 2026-02-24T18:10:39.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:457: TEST_divergent_2: non_divergent=' 0' 2026-02-24T18:10:39.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:452: TEST_divergent_2: for i in $osds 2026-02-24T18:10:39.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:454: TEST_divergent_2: '[' 1 = 1 ']' 2026-02-24T18:10:39.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:455: TEST_divergent_2: continue 2026-02-24T18:10:39.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:452: TEST_divergent_2: for i in $osds 2026-02-24T18:10:39.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:454: TEST_divergent_2: '[' 2 = 1 ']' 2026-02-24T18:10:39.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:457: TEST_divergent_2: non_divergent=' 0 2' 2026-02-24T18:10:39.382 INFO:tasks.workunit.client.0.trial172.stdout:writing initial objects 2026-02-24T18:10:39.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:460: TEST_divergent_2: echo 'writing initial objects' 2026-02-24T18:10:39.382 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: seq 1 100 2026-02-24T18:10:39.383 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.383 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_1 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.406 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.406 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_2 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.437 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.437 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_3 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.465 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.466 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_4 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.494 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.494 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_5 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.515 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.515 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_6 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.544 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.544 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_7 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.570 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.570 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_8 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.595 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_9 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.623 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.623 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_10 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.652 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.652 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_11 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.673 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.673 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_12 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.701 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.701 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_13 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.726 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.726 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_14 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.755 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.755 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_15 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.776 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.776 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_16 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.804 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.805 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_17 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.833 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.833 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_18 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.856 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.856 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_19 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.878 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.878 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_20 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.907 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.908 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_21 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.934 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.934 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_22 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.953 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.953 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_23 /tmp/tmp.AohTN555hP 2026-02-24T18:10:39.982 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:39.982 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_24 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.009 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.009 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_25 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.030 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.030 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_26 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_27 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.080 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.080 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_28 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.106 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.106 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_29 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.131 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.131 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_30 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.159 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.159 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_31 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_32 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.206 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.207 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_33 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.228 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.228 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_34 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_35 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_36 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.305 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.305 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_37 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.332 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.332 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_38 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.352 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.353 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_39 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_40 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.402 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.402 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_41 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_42 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.457 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.457 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_43 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_44 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.512 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.512 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_45 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_46 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.568 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.568 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_47 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.593 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_48 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.621 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.621 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_49 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.649 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.649 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_50 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.677 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.677 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_51 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.704 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.704 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_52 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.733 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.733 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_53 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_54 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.789 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.789 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_55 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.817 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.817 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_56 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_57 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.873 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.874 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_58 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.901 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.902 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_59 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.929 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.929 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_60 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.956 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.956 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_61 /tmp/tmp.AohTN555hP 2026-02-24T18:10:40.984 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:40.985 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_62 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.012 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.013 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_63 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.037 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.037 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_64 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.065 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.066 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_65 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.093 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.093 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_66 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.121 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.121 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_67 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.149 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.149 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_68 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.177 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.177 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_69 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.205 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.205 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_70 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.234 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.234 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_71 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_72 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.289 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.289 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_73 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.317 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.317 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_74 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.345 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.345 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_75 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_76 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.402 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.402 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_77 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_78 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.457 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.457 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_79 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_80 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.512 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.513 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_81 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_82 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.569 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.569 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_83 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_84 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.625 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.626 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_85 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.654 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.654 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_86 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.682 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.682 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_87 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.710 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_88 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.737 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.737 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_89 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.765 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.765 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_90 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.792 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.792 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_91 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_92 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.849 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.849 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_93 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.877 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.878 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_94 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.906 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.906 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_95 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_96 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.962 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.962 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_97 /tmp/tmp.AohTN555hP 2026-02-24T18:10:41.992 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:41.992 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_98 /tmp/tmp.AohTN555hP 2026-02-24T18:10:42.021 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:42.021 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_99 /tmp/tmp.AohTN555hP 2026-02-24T18:10:42.041 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-24T18:10:42.041 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_100 /tmp/tmp.AohTN555hP 2026-02-24T18:10:42.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:467: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-24T18:10:42.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:467: TEST_divergent_2: wait_for_clean 2026-02-24T18:10:42.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:10:42.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:10:42.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:10:42.071 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-24T18:10:42.071 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:10:42.071 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:10:42.072 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:10:42.072 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:10:42.072 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:10:42.142 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-24T18:10:42.142 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:10:42.142 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:10:42.142 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:10:42.142 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:10:42.142 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:10:42.366 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:10:42.367 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:10:42.367 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:10:42.367 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:10:42.367 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:10:42.367 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:10:42.448 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836490 2026-02-24T18:10:42.448 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836490 2026-02-24T18:10:42.448 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490' 2026-02-24T18:10:42.448 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:10:42.449 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:10:42.532 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672969 2026-02-24T18:10:42.532 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672969 2026-02-24T18:10:42.532 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-42949672969' 2026-02-24T18:10:42.532 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:10:42.532 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:10:42.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=60129542153 2026-02-24T18:10:42.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 60129542153 2026-02-24T18:10:42.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-42949672969 2-60129542153' 2026-02-24T18:10:42.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:10:42.617 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836490 2026-02-24T18:10:42.617 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:10:42.619 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:10:42.619 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836490 2026-02-24T18:10:42.619 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:10:42.621 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836490 2026-02-24T18:10:42.621 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 21474836490 2026-02-24T18:10:42.621 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-24T18:10:42.621 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:10:42.839 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-24T18:10:42.839 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:10:43.841 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:10:43.841 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:10:44.064 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836490 -lt 21474836490 2026-02-24T18:10:44.064 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:10:44.065 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672969 2026-02-24T18:10:44.065 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:10:44.066 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:10:44.067 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672969 2026-02-24T18:10:44.067 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:10:44.068 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672969 2026-02-24T18:10:44.068 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 42949672969 2026-02-24T18:10:44.069 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672969' 2026-02-24T18:10:44.069 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:10:44.283 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672969 -lt 42949672969 2026-02-24T18:10:44.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:10:44.284 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-60129542153 2026-02-24T18:10:44.285 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:10:44.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:10:44.286 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-60129542153 2026-02-24T18:10:44.286 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:10:44.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=60129542153 2026-02-24T18:10:44.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-24T18:10:44.288 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 60129542153 2026-02-24T18:10:44.288 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:10:44.518 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-24T18:10:44.519 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:10:44.519 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:10:44.519 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:10:44.814 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:10:44.815 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:10:44.815 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:10:44.815 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:10:44.815 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:10:44.815 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:10:44.815 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:10:44.815 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:10:45.031 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-24T18:10:45.031 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:10:45.031 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:10:45.031 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:10:45.321 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-24T18:10:45.322 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T18:10:45.322 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T18:10:45.322 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:469: TEST_divergent_2: get_pg test existing_1 2026-02-24T18:10:45.322 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=test 2026-02-24T18:10:45.322 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=existing_1 2026-02-24T18:10:45.323 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map test existing_1 2026-02-24T18:10:45.323 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-24T18:10:45.550 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:469: TEST_divergent_2: local pgid=1.0 2026-02-24T18:10:45.550 INFO:tasks.workunit.client.0.trial172.stdout:blackholing osds 0 2 2026-02-24T18:10:45.550 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:472: TEST_divergent_2: echo 'blackholing osds 0 2' 2026-02-24T18:10:45.550 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:473: TEST_divergent_2: ceph pg dump pgs 2026-02-24T18:10:45.751 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:10:45.751 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 0 0 0 51200 0 0 16 84 16 active+clean 2026-02-24T18:10:33.767543+0000 27'100 27:122 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:10:32.829947+0000 0'0 2026-02-24T18:10:32.829947+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:56:53.688698+0000 0 0 2026-02-24T18:10:45.752 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:10:45.752 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:10:45.752 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:10:45.760 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:474: TEST_divergent_2: for i in $non_divergent 2026-02-24T18:10:45.767 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:476: TEST_divergent_2: get_asok_path osd.0 2026-02-24T18:10:45.767 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T18:10:45.767 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T18:10:45.767 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:10:45.767 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:10:45.767 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:10:45.767 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.0.asok 2026-02-24T18:10:45.767 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:476: TEST_divergent_2: CEPH_ARGS= 2026-02-24T18:10:45.767 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:476: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-24T18:10:45.815 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:10:45.815 INFO:tasks.workunit.client.0.trial172.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-24T18:10:45.815 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:10:45.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:474: TEST_divergent_2: for i in $non_divergent 2026-02-24T18:10:45.821 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:476: TEST_divergent_2: get_asok_path osd.2 2026-02-24T18:10:45.822 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-24T18:10:45.822 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-24T18:10:45.822 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:10:45.822 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:10:45.822 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:10:45.822 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.2.asok 2026-02-24T18:10:45.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:476: TEST_divergent_2: CEPH_ARGS= 2026-02-24T18:10:45.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:476: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-24T18:10:45.874 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:10:45.874 INFO:tasks.workunit.client.0.trial172.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-24T18:10:45.874 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:10:45.879 INFO:tasks.workunit.client.0.trial172.stdout:create new divergent objects 2026-02-24T18:10:45.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:480: TEST_divergent_2: echo 'create new divergent objects' 2026-02-24T18:10:45.880 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:481: TEST_divergent_2: seq 1 2 2026-02-24T18:10:45.881 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:481: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-24T18:10:45.881 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:481: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-24T18:10:45.881 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:483: TEST_divergent_2: rados -p test create newobject_1 2026-02-24T18:10:45.881 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:486: TEST_divergent_2: echo 'writing divergent objects' 2026-02-24T18:10:45.882 INFO:tasks.workunit.client.0.trial172.stdout:writing divergent objects 2026-02-24T18:10:45.882 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:483: TEST_divergent_2: rados -p test create newobject_2 2026-02-24T18:10:45.883 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:487: TEST_divergent_2: seq 1 2 2026-02-24T18:10:45.883 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:487: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-24T18:10:45.883 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:487: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-24T18:10:45.883 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:489: TEST_divergent_2: rados -p test put existing_1 /tmp/tmp.OgRbPGtOtd 2026-02-24T18:10:45.884 INFO:tasks.workunit.client.0.trial172.stdout:remove divergent objects 2026-02-24T18:10:45.885 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:492: TEST_divergent_2: echo 'remove divergent objects' 2026-02-24T18:10:45.885 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:489: TEST_divergent_2: rados -p test put existing_2 /tmp/tmp.OgRbPGtOtd 2026-02-24T18:10:45.885 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:493: TEST_divergent_2: seq 1 2 2026-02-24T18:10:45.885 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:493: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-24T18:10:45.885 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: expr 1 + 2 2026-02-24T18:10:45.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: rmi=3 2026-02-24T18:10:45.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:493: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-24T18:10:45.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:496: TEST_divergent_2: rados -p test rm existing_3 2026-02-24T18:10:45.887 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: expr 2 + 2 2026-02-24T18:10:45.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: rmi=4 2026-02-24T18:10:45.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:498: TEST_divergent_2: sleep 10 2026-02-24T18:10:45.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:496: TEST_divergent_2: rados -p test rm existing_4 2026-02-24T18:10:55.891 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:499: TEST_divergent_2: killall -9 rados 2026-02-24T18:10:55.946 INFO:tasks.workunit.client.0.trial172.stderr:rados: no process found 2026-02-24T18:10:55.946 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:502: TEST_divergent_2: echo 'killing all the osds' 2026-02-24T18:10:55.946 INFO:tasks.workunit.client.0.trial172.stdout:killing all the osds 2026-02-24T18:10:55.947 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:503: TEST_divergent_2: ceph pg dump pgs 2026-02-24T18:10:56.162 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:10:56.163 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 0 0 0 51152 0 0 16 90 16 active+clean 2026-02-24T18:10:33.767543+0000 27'106 27:128 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:10:32.829947+0000 0'0 2026-02-24T18:10:32.829947+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:56:53.688698+0000 0 0 2026-02-24T18:10:56.163 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:10:56.163 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:10:56.163 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:10:56.172 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:504: TEST_divergent_2: kill_daemons td/divergent-priors KILL osd 2026-02-24T18:10:56.173 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:10:56.173 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:10:56.173 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:10:56.173 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:10:56.173 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:10:56.282 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:10:56.283 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:505: TEST_divergent_2: for i in $osds 2026-02-24T18:10:56.283 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:507: TEST_divergent_2: ceph osd down osd.0 2026-02-24T18:10:56.550 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already down. 2026-02-24T18:10:56.560 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:505: TEST_divergent_2: for i in $osds 2026-02-24T18:10:56.560 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:507: TEST_divergent_2: ceph osd down osd.1 2026-02-24T18:10:56.839 INFO:tasks.workunit.client.0.trial172.stderr:osd.1 is already down. 2026-02-24T18:10:56.851 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:505: TEST_divergent_2: for i in $osds 2026-02-24T18:10:56.851 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:507: TEST_divergent_2: ceph osd down osd.2 2026-02-24T18:10:57.145 INFO:tasks.workunit.client.0.trial172.stderr:osd.2 is already down. 2026-02-24T18:10:57.154 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:509: TEST_divergent_2: for i in $non_divergent 2026-02-24T18:10:57.155 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:511: TEST_divergent_2: ceph osd out osd.0 2026-02-24T18:10:57.437 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already out. 2026-02-24T18:10:57.447 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:509: TEST_divergent_2: for i in $non_divergent 2026-02-24T18:10:57.447 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:511: TEST_divergent_2: ceph osd out osd.2 2026-02-24T18:10:57.733 INFO:tasks.workunit.client.0.trial172.stderr:osd.2 is already out. 2026-02-24T18:10:57.743 INFO:tasks.workunit.client.0.trial172.stdout:bringing up non_divergent 0 2 2026-02-24T18:10:57.743 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:515: TEST_divergent_2: echo 'bringing up non_divergent 0 2' 2026-02-24T18:10:57.743 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:516: TEST_divergent_2: ceph pg dump pgs 2026-02-24T18:10:57.955 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:10:57.956 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 0 0 0 51152 0 0 16 90 16 stale+active+clean 2026-02-24T18:10:33.767543+0000 27'106 27:128 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:10:32.829947+0000 0'0 2026-02-24T18:10:32.829947+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:56:53.688698+0000 0 0 2026-02-24T18:10:57.956 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:10:57.956 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:10:57.956 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:10:57.969 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:517: TEST_divergent_2: for i in $non_divergent 2026-02-24T18:10:57.969 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:519: TEST_divergent_2: activate_osd td/divergent-priors 0 2026-02-24T18:10:57.970 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:10:57.970 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:10:57.970 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-24T18:10:57.970 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:10:57.970 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/0 2026-02-24T18:10:57.970 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:10:57.970 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:10:57.970 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:10:57.970 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:10:57.970 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-24T18:10:57.971 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-24T18:10:57.971 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:10:57.971 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:10:57.971 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:10:57.971 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:10:57.971 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:10:57.971 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:10:57.971 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:10:57.971 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:10:57.971 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:10:57.972 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:10:57.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:10:57.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:10:57.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:10:57.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:10:57.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:10:57.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:10:57.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:10:57.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:10:57.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:10:57.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:10:57.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/0 2026-02-24T18:10:57.973 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:10:57.974 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-24T18:10:57.974 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:10:57.974 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/0/whoami 2026-02-24T18:10:57.974 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-24T18:10:57.975 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:10:57.975 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:10:57.975 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:10:57.993 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:57.979+0000 7f1588f0fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:57.994 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:57.983+0000 7f1588f0fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:57.995 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:57.983+0000 7f1588f0fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:58.186 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-24T18:10:58.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:10:58.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:10:58.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:10:58.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:10:58.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:10:58.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:10:58.187 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:10:58.188 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:10:58.188 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:10:58.408 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:58.394+0000 7f1588f0fa40 -1 Falling back to public interface 2026-02-24T18:10:58.411 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:10:59.088 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:59.074+0000 7f1588f0fa40 -1 osd.0 27 log_to_monitors true 2026-02-24T18:10:59.412 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:10:59.413 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:10:59.413 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:10:59.413 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:10:59.414 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:10:59.414 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:10:59.639 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up out weight 0 up_from 34 up_thru 0 down_at 28 last_clean_interval [5,27) [v2:127.0.0.1:6800/1761963574,v1:127.0.0.1:6801/1761963574] [v2:127.0.0.1:6802/1761963574,v1:127.0.0.1:6803/1761963574] exists,up 0d39d29d-6e3b-4fa3-a91e-35af674e24c9 2026-02-24T18:10:59.642 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:10:59.642 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:10:59.643 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:10:59.643 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:517: TEST_divergent_2: for i in $non_divergent 2026-02-24T18:10:59.643 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:519: TEST_divergent_2: activate_osd td/divergent-priors 2 2026-02-24T18:10:59.643 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:10:59.643 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:10:59.643 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T18:10:59.643 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:10:59.643 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/2 2026-02-24T18:10:59.643 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:10:59.643 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:10:59.644 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:10:59.644 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:10:59.644 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-24T18:10:59.644 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-24T18:10:59.644 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:10:59.644 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:10:59.644 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:10:59.644 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:10:59.644 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:10:59.644 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:10:59.644 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:10:59.644 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:10:59.644 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:10:59.645 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:10:59.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:10:59.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:10:59.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:10:59.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:10:59.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:10:59.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:10:59.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:10:59.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:10:59.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:10:59.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:10:59.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/2 2026-02-24T18:10:59.646 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:10:59.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T18:10:59.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:10:59.647 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/2/whoami 2026-02-24T18:10:59.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T18:10:59.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:10:59.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:10:59.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:10:59.665 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:59.650+0000 7ff487bb3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:59.666 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:59.654+0000 7ff487bb3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:59.667 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:10:59.654+0000 7ff487bb3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:10:59.859 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T18:10:59.859 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:10:59.859 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:10:59.859 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:10:59.859 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:10:59.859 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:10:59.860 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:10:59.860 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:10:59.860 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:10:59.861 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:11:00.083 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:11:00.095 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:11:00.082+0000 7ff487bb3a40 -1 Falling back to public interface 2026-02-24T18:11:00.730 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:11:00.714+0000 7ff487bb3a40 -1 osd.2 27 log_to_monitors true 2026-02-24T18:11:01.085 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:11:01.085 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:11:01.085 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:11:01.086 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:11:01.086 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:11:01.086 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:11:01.306 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up out weight 0 up_from 36 up_thru 0 down_at 30 last_clean_interval [14,27) [v2:127.0.0.1:6810/3266666167,v1:127.0.0.1:6811/3266666167] [v2:127.0.0.1:6812/3266666167,v1:127.0.0.1:6813/3266666167] exists,up aae44b3b-9b8d-43f3-925b-7313233d9556 2026-02-24T18:11:01.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:11:01.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:11:01.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:11:01.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:521: TEST_divergent_2: for i in $non_divergent 2026-02-24T18:11:01.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:523: TEST_divergent_2: ceph osd in osd.0 2026-02-24T18:11:01.579 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already in. 2026-02-24T18:11:01.589 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:521: TEST_divergent_2: for i in $non_divergent 2026-02-24T18:11:01.589 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:523: TEST_divergent_2: ceph osd in osd.2 2026-02-24T18:11:01.855 INFO:tasks.workunit.client.0.trial172.stderr:osd.2 is already in. 2026-02-24T18:11:01.865 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:526: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-24T18:11:01.865 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:526: TEST_divergent_2: wait_for_clean 2026-02-24T18:11:01.865 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:11:01.866 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:11:01.866 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:11:01.866 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-24T18:11:01.867 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:11:01.867 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:11:01.867 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:11:01.867 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:11:01.867 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:11:01.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-24T18:11:01.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:11:01.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:11:01.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:11:01.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:11:01.931 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:11:02.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:11:02.151 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:11:02.151 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:11:02.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:11:02.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:11:02.151 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:11:02.237 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=146028888067 2026-02-24T18:11:02.237 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 146028888067 2026-02-24T18:11:02.237 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-146028888067' 2026-02-24T18:11:02.237 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:11:02.237 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:11:02.298 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq= 2026-02-24T18:11:02.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: true 2026-02-24T18:11:02.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z '' 2026-02-24T18:11:02.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: flush_pg_stats: continue 2026-02-24T18:11:02.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:11:02.299 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:11:02.379 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=154618822659 2026-02-24T18:11:02.379 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 154618822659 2026-02-24T18:11:02.379 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-146028888067 2-154618822659' 2026-02-24T18:11:02.379 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:11:02.380 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-146028888067 2026-02-24T18:11:02.380 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:11:02.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:11:02.382 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-146028888067 2026-02-24T18:11:02.382 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:11:02.383 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=146028888067 2026-02-24T18:11:02.383 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 146028888067' 2026-02-24T18:11:02.383 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 146028888067 2026-02-24T18:11:02.383 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:11:02.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 1 -lt 146028888067 2026-02-24T18:11:02.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:11:03.605 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:11:03.606 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:11:03.826 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 146028888067 -lt 146028888067 2026-02-24T18:11:03.826 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:11:03.826 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-154618822659 2026-02-24T18:11:03.827 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:11:03.827 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:11:03.828 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-154618822659 2026-02-24T18:11:03.828 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:11:03.830 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=154618822659 2026-02-24T18:11:03.830 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 154618822659 2026-02-24T18:11:03.830 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 154618822659' 2026-02-24T18:11:03.831 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:11:04.061 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 154618822659 -lt 154618822659 2026-02-24T18:11:04.062 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:11:04.062 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:04.062 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:04.356 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:11:04.357 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:04.358 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:04.358 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:04.358 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:04.358 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:04.358 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:04.358 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:04.571 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:04.572 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:04.572 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:04.572 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:04.867 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:04.867 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-24T18:11:04.867 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-24T18:11:04.867 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-24T18:11:04.867 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:04.867 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-24T18:11:04.968 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:11:04.968 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:04.969 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:04.969 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:04.969 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:04.970 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:04.970 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:04.970 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:05.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:05.188 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:05.188 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:05.188 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:05.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:05.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:11:05.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:11:05.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:11:05.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:11:05.475 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:11:05.475 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:11:05.475 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:11:05.475 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:11:05.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:11:05.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:11:05.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 8 )) 2026-02-24T18:11:05.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:05.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-24T18:11:05.962 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:11:05.963 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:05.964 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:05.964 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:05.964 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:05.964 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:05.964 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:05.964 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:06.181 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:06.182 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:06.182 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:06.182 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:06.473 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:06.473 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:11:06.473 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:11:06.473 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:11:06.473 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:11:06.473 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:11:06.473 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:11:06.474 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:11:06.474 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:11:06.759 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:11:06.759 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:11:06.759 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 8 )) 2026-02-24T18:11:06.759 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:06.759 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-24T18:11:07.160 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:11:07.160 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:07.161 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:07.161 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:07.162 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:07.162 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:07.162 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:07.162 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:07.377 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:07.377 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:07.378 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:07.378 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:07.672 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:07.672 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:11:07.672 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:11:07.672 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:11:07.672 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:11:07.672 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:11:07.672 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:11:07.673 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:11:07.673 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:11:07.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:11:07.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:11:07.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 8 )) 2026-02-24T18:11:07.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:07.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-24T18:11:08.772 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:11:08.772 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:08.773 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:08.773 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:08.773 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:08.773 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:08.773 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:08.773 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:08.993 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:08.994 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:08.994 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:08.994 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:09.280 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:09.280 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:11:09.280 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:11:09.280 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:11:09.280 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:11:09.280 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:11:09.280 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:11:09.280 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:11:09.280 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:11:09.565 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:11:09.565 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:11:09.565 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 8 )) 2026-02-24T18:11:09.565 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:09.566 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 1.6 2026-02-24T18:11:11.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:11:11.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:11.167 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:11.167 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:11.167 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:11.167 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:11.167 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:11.167 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:11.377 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:11.377 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:11.378 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:11.378 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:11.655 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:11.655 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:11:11.655 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:11:11.655 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:11:11.655 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:11:11.655 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:11:11.656 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:11:11.656 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:11:11.656 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:11:11.936 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:11:11.936 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:11:11.936 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 5 >= 8 )) 2026-02-24T18:11:11.936 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:11.936 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 3.2 2026-02-24T18:11:15.137 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:11:15.137 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:15.138 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:15.138 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:15.138 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:15.138 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:15.138 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:15.139 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:15.352 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:15.352 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:15.353 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:15.353 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:15.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:15.646 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:11:15.646 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:11:15.646 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:11:15.646 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:11:15.646 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:11:15.646 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:11:15.646 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:11:15.647 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:11:15.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:11:15.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:11:15.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 6 >= 8 )) 2026-02-24T18:11:15.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:15.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 6.4 2026-02-24T18:11:22.340 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:11:22.340 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:22.341 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:22.342 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:22.342 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:22.342 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:22.342 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:22.342 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:22.569 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:22.570 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:22.570 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:22.570 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:22.852 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:22.852 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:11:22.852 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:11:22.852 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:11:22.852 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:11:22.852 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:11:22.852 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:11:22.853 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:11:22.853 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:11:23.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:11:23.139 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:11:23.139 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 7 >= 8 )) 2026-02-24T18:11:23.139 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:23.139 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 7.3 2026-02-24T18:11:30.440 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:11:30.440 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:30.441 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:30.441 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:30.441 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:30.441 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:30.441 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:30.441 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:30.652 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:30.653 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:30.653 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:30.653 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:30.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:30.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:11:30.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:11:30.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:11:30.945 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:11:30.945 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:11:30.945 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:11:30.945 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:11:30.945 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:11:31.239 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:11:31.239 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:11:31.239 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 8 >= 8 )) 2026-02-24T18:11:31.239 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_fingerprint": "e511c6b5-e915-4e2d-bfe4-1d7c43be13b7", 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "version": "20.3.0-5507-g618c318a", 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "commit": "618c318a7a0b287ab5aeff207b7bf1d73ef45044", 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "timestamp": "2026-02-24T18:11:31.498759+0000", 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "tag": "", 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "health": { 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "status": "HEALTH_WARN", 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "checks": { 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "OSDMAP_FLAGS": { 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "count": 17 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [], 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "OSD_DOWN": { 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:11:31.511 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "message": "1 osds down", 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "count": 1 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [ 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "message": "osd.1 (root=default,host=trial172) is down" 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "PG_DEGRADED": { 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "count": 1 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [ 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "mutes": [] 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "monmap_first_committed": 1, 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "monmap_last_committed": 1, 2026-02-24T18:11:31.512 INFO:tasks.workunit.client.0.trial172.stdout: "monmap": { 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 1, 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "fsid": "27512777-e37f-4f16-b176-1621842bd4df", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:10:20.627140Z", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "created": "2026-02-24T18:10:20.627140Z", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "min_mon_release": 20, 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "min_mon_release_name": "tentacle", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "election_strategy": 1, 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "disallowed_leaders": "", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode": false, 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "tiebreaker_mon": "", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "removed_ranks": "", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "features": { 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "persistent": [ 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "kraken", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "luminous", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "mimic", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap-prune", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "nautilus", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "octopus", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "pacific", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "elector-pinging", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "quincy", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "reef", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "squid", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "tentacle", 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "nvmeof_beacon_diff" 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "optional": [] 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: "mons": [ 2026-02-24T18:11:31.513 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "rank": 0, 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "name": "a", 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:7115", 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 0 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:7115/0", 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "priority": 0, 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 0, 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "time_added": "2026-02-24T18:10:20:627138+0000", 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "crush_location": [] 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "quorum": [ 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: 0 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap": { 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 39, 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "fsid": "27512777-e37f-4f16-b176-1621842bd4df", 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "created": "2026-02-24T18:10:20.710665+0000", 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:11:01.886611+0000", 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "last_up_change": "2026-02-24T18:11:00.876859+0000", 2026-02-24T18:11:31.514 INFO:tasks.workunit.client.0.trial172.stdout: "last_in_change": "2026-02-24T18:11:01.785925+0000", 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "flags_num": 5800384, 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "flags_set": [ 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "nodown", 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "noin", 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "noout", 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "pglog_hardlimit", 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snapdirs", 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "recovery_deletes", 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "sortbitwise" 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "crush_version": 9, 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "full_ratio": 0.99000000953674316, 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_snapshot": "", 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "pool_max": 1, 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "max_osd": 3, 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "require_min_compat_client": "luminous", 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "min_compat_client": "jewel", 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "require_osd_release": "tentacle", 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "allow_crimson": false, 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "pools": [ 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "pool": 1, 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "pool_name": "test", 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "create_time": "2026-02-24T18:10:32.804007+0000", 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "flags": 1, 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "flags_names": "hashpspool", 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "type": 1, 2026-02-24T18:11:31.515 INFO:tasks.workunit.client.0.trial172.stdout: "size": 3, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "min_size": 2, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "crush_rule": 0, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_count": 0, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_target": 0, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_barrier": 0, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "is_stretch_pool": false, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "object_hash": 2, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "pg_autoscale_mode": "off", 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num": 1, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "pg_placement_num": 1, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "pg_placement_num_target": 1, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num_target": 1, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num_pending": 1, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "last_pg_merge_meta": { 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "source_pgid": "0.0", 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "ready_epoch": 0, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_started": 0, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_clean": 0, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "source_version": "0'0", 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "target_version": "0'0" 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "last_change": "27", 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend": "0", 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend_preluminous": "0", 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "auid": 0, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "snap_mode": "selfmanaged", 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "snap_seq": 0, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "snap_epoch": 0, 2026-02-24T18:11:31.516 INFO:tasks.workunit.client.0.trial172.stdout: "pool_snaps": [], 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "removed_snaps": "[]", 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "quota_max_bytes": 0, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "quota_max_objects": 0, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "tiers": [], 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "tier_of": -1, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "read_tier": -1, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "write_tier": -1, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "cache_mode": "none", 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "target_max_bytes": 0, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "target_max_objects": 0, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "cache_min_flush_age": 0, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "cache_min_evict_age": 0, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "erasure_code_profile": "", 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_params": { 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "type": "none" 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_period": 0, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_count": 0, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "use_gmt_hitset": true, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "min_read_recency_for_promote": 0, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "min_write_recency_for_promote": 0, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_grade_decay_rate": 0, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_search_last_n": 0, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "grade_table": [], 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "stripe_width": 0, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "expected_num_objects": 0, 2026-02-24T18:11:31.517 INFO:tasks.workunit.client.0.trial172.stdout: "fast_read": false, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "nonprimary_shards": "{}", 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "options": {}, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "application_metadata": {}, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "read_balance": { 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "score_type": "Fair distribution", 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "score_acting": 2.9900000095367432, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "score_stable": 2.9900000095367432, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "optimal_score": 0.67000001668930054, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "raw_score_acting": 2, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "raw_score_stable": 2, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity_weighted": 1, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "average_primary_affinity": 1, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "average_primary_affinity_weighted": 1 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "osds": [ 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "0d39d29d-6e3b-4fa3-a91e-35af674e24c9", 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "up": 1, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 5, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 27, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 34, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 38, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 28, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:11:31.518 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6800", 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6801", 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6802", 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6803", 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:11:31.519 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6806", 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6807", 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6804", 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6805", 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6801/1761963574", 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6803/1761963574", 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6807/1761963574", 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6805/1761963574", 2026-02-24T18:11:31.520 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "exists", 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "up" 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "d7cb9f1a-c825-4426-aded-3118e4460724", 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "up": 0, 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 0, 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 0, 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 10, 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 21, 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 29, 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6810", 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6811", 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.521 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6812", 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6813", 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6816", 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6817", 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:11:31.522 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6814", 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6815", 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6811/3631029146", 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6813/3631029146", 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3631029146", 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3631029146", 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "exists" 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "aae44b3b-9b8d-43f3-925b-7313233d9556", 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "up": 1, 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 14, 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 27, 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 36, 2026-02-24T18:11:31.523 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 0, 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 30, 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6810", 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6811", 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6812", 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6813", 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.524 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6816", 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6817", 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6814", 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6815", 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6811/3266666167", 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6813/3266666167", 2026-02-24T18:11:31.525 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3266666167", 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3266666167", 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "exists", 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "up" 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "osd_xinfo": [ 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:10:56.483449+0000", 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:10:24.418099+0000", 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:10:56.774679+0000", 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:11:31.526 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:10:57.078386+0000", 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:10:30.952922+0000", 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap": [], 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap_items": [], 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap_primaries": [], 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "pg_temp": [], 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "primary_temp": [], 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "blocklist": {}, 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "range_blocklist": {}, 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "erasure_code_profiles": { 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "default": { 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "k": "2", 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "m": "2", 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "plugin": "isa", 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "technique": "reed_sol_van" 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "removed_snaps_queue": [], 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "new_removed_snaps": [], 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "new_purged_snaps": [], 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "crush_node_flags": {}, 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "device_class_flags": {}, 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode": { 2026-02-24T18:11:31.527 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode_enabled": false, 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_bucket_count": 0, 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "degraded_stretch_mode": 0, 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "recovering_stretch_mode": 0, 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode_bucket": 0 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "osd_metadata": [ 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6802/1761963574,v1:127.0.0.1:6803/1761963574]", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:11:31.528 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:10:21.741311Z", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6800/1761963574,v1:127.0.0.1:6801/1761963574]", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/1761963574,v1:127.0.0.1:6807/1761963574]", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/1761963574,v1:127.0.0.1:6805/1761963574]", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/0", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:11:31.529 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6812/3631029146,v1:127.0.0.1:6813/3631029146]", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:11:31.530 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:10:25.552571Z", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6810/3631029146,v1:127.0.0.1:6811/3631029146]", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3631029146,v1:127.0.0.1:6817/3631029146]", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3631029146,v1:127.0.0.1:6815/3631029146]", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/1", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.531 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6812/3266666167,v1:127.0.0.1:6813/3266666167]", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:10:28.432213Z", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:11:31.532 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6810/3266666167,v1:127.0.0.1:6811/3266666167]", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3266666167,v1:127.0.0.1:6817/3266666167]", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3266666167,v1:127.0.0.1:6815/3266666167]", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/2", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_clean_epochs": { 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "min_last_epoch_clean": 0, 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_clean": { 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "per_pool": [] 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.533 INFO:tasks.workunit.client.0.trial172.stdout: "osd_epochs": [ 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 34 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 37 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_first_committed": 1, 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_last_committed": 39, 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "crushmap": { 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "devices": [ 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.0", 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.1", 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.2", 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.534 INFO:tasks.workunit.client.0.trial172.stdout: "types": [ 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 0, 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd" 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "name": "host" 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 2, 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "name": "chassis" 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 3, 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "name": "rack" 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 4, 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "name": "row" 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 5, 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "name": "pdu" 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 6, 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "name": "pod" 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 7, 2026-02-24T18:11:31.535 INFO:tasks.workunit.client.0.trial172.stdout: "name": "room" 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 8, 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "name": "datacenter" 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 9, 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "name": "zone" 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 10, 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "name": "region" 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "name": "root" 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "buckets": [ 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "id": -1, 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "name": "default", 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "root", 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "id": -3, 2026-02-24T18:11:31.536 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "id": -2, 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "name": "default~ssd", 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "root", 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "id": -4, 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "id": -3, 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "name": "trial172", 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "host", 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:11:31.537 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 1 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 2 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "id": -4, 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "name": "trial172~ssd", 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "host", 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.538 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 1 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 2 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "rules": [ 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "rule_id": 0, 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "rule_name": "replicated_rule", 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "type": 1, 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "steps": [ 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "op": "take", 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "item": -1, 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "item_name": "default" 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "op": "choose_firstn", 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "num": 0, 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "type": "osd" 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: "op": "emit" 2026-02-24T18:11:31.539 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "tunables": { 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "choose_local_tries": 0, 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "choose_local_fallback_tries": 0, 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "choose_total_tries": 50, 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_descend_once": 1, 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_vary_r": 1, 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_stable": 1, 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "msr_descents": 100, 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "msr_collision_tries": 100, 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "straw_calc_version": 1, 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "allowed_bucket_algs": 54, 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stderr:report 2234768674 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "profile": "jewel", 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "optimal_tunables": 1, 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "legacy_tunables": 0, 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "minimum_required_version": "jewel", 2026-02-24T18:11:31.540 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables": 1, 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables2": 1, 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "has_v2_rules": 0, 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables3": 1, 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "has_v3_rules": 0, 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "has_v4_buckets": 1, 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables5": 1, 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "has_v5_rules": 0, 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "has_msr_rules": 0 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "choose_args": {} 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "fsmap": { 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 1, 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "btime": "2026-02-24T18:10:20:710433+0000", 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "default_fscid": -1, 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "compat": { 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "compat": {}, 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "ro_compat": {}, 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "incompat": { 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "feature_1": "base v0.20", 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "feature_2": "client writeable ranges", 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "feature_3": "default file layouts on dirs", 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "feature_4": "dir inode in separate object", 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "feature_5": "mds uses versioned encoding", 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "feature_8": "no anchor table", 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "feature_9": "file layout v2", 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "feature_10": "snaprealm v2", 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "feature_11": "minor log segments", 2026-02-24T18:11:31.541 INFO:tasks.workunit.client.0.trial172.stdout: "feature_12": "quiesce subvolumes" 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "feature_flags": { 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "enable_multiple": true, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "ever_enabled_multiple": true 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "standbys": [], 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "filesystems": [] 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "mdsmap_first_committed": 1, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "mdsmap_last_committed": 1, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "auth": { 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "first_committed": 1, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "last_committed": 5, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "num_secrets": 3 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg": 1, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_active": 1, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_unknown": 0, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "num_osd": 3, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "pool_sum": { 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "stat_sum": { 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes": 51200, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects": 100, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_clones": 0, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_copies": 300, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing": 0, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_degraded": 100, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_misplaced": 0, 2026-02-24T18:11:31.542 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_unfound": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_dirty": 100, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_whiteouts": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_read": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_read_kb": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_write": 100, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_write_kb": 100, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_scrub_errors": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_deep_scrub_errors": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_recovered": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_recovered": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_keys_recovered": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_omap": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_kb": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_kb": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_promote": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_high": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_low": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_some": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_full": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_pinned": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_legacy_snapsets": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_large_omap_objects": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_manifest": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_bytes": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_keys": 0, 2026-02-24T18:11:31.543 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_repaired": 0 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "store_stats": { 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "total": 0, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "available": 0, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 0, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 0, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 0, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 0 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "log_size": 16, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "ondisk_log_size": 16, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "up": 2, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "acting": 2, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "num_store_stats": 0 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "osd_sum": { 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 0, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 0, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "num_pgs": 3, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "num_osds": 3, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_osds": 3, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_omap_osds": 3, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "kb": 314572800, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used": 83840, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_data": 2056, 2026-02-24T18:11:31.544 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_omap": 24, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_meta": 81703, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "kb_avail": 314488960, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "statfs": { 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "total": 322122547200, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "available": 322036695040, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 2105344, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 414763, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 25401, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 83664071 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "hb_peers": [], 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "snap_trim_queue_len": 0, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "num_snap_trimming": 0, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "num_shards_repaired": 0, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "op_queue_age_hist": { 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "histogram": [], 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "upper_bound": 1 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "perf_stat": { 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ms": 0, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ms": 0, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ns": 0, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ns": 0 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "alerts": [], 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: "network_ping_times": [] 2026-02-24T18:11:31.545 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "osd_sum_by_class": { 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "ssd": { 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 0, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 0, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "num_pgs": 3, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "num_osds": 3, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_osds": 3, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_omap_osds": 3, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "kb": 314572800, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used": 83840, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_data": 2056, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_omap": 24, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_meta": 81703, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "kb_avail": 314488960, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "statfs": { 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "total": 322122547200, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "available": 322036695040, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 2105344, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 414763, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 25401, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 83664071 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "hb_peers": [], 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "snap_trim_queue_len": 0, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "num_snap_trimming": 0, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "num_shards_repaired": 0, 2026-02-24T18:11:31.546 INFO:tasks.workunit.client.0.trial172.stdout: "op_queue_age_hist": { 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "histogram": [], 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "upper_bound": 1 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "perf_stat": { 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ms": 0, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ms": 0, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ns": 0, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ns": 0 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "alerts": [], 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "network_ping_times": [] 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "pool_stats": [ 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": 1, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg": 1, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "stat_sum": { 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes": 51200, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects": 100, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_clones": 0, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_copies": 300, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing": 0, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_degraded": 100, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_misplaced": 0, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_unfound": 0, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_dirty": 100, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "num_whiteouts": 0, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "num_read": 0, 2026-02-24T18:11:31.547 INFO:tasks.workunit.client.0.trial172.stdout: "num_read_kb": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_write": 100, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_write_kb": 100, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_scrub_errors": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_deep_scrub_errors": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_recovered": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_recovered": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_keys_recovered": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_omap": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_kb": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_kb": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_promote": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_high": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_low": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_some": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_full": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_pinned": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_legacy_snapsets": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_large_omap_objects": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_manifest": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_bytes": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_keys": 0, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_repaired": 0 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "store_stats": { 2026-02-24T18:11:31.548 INFO:tasks.workunit.client.0.trial172.stdout: "total": 0, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "available": 0, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 1220608, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 153552, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 0, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 0 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "log_size": 16, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "ondisk_log_size": 16, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "up": 2, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "acting": 2, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "num_store_stats": 3 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "osd_stats": [ 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 146028888073 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 42949672972 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 154618822665 2026-02-24T18:11:31.549 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_by_state": [ 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "state": "active+undersized+degraded", 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "num": 1 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "pool_pg_unavailable_map": [ 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": "1", 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "pgs": [] 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_by_osd": [ 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 1, 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 1, 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 1, 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:11:31.550 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 0, 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snaps": [ 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "pool": 1, 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snaps": {} 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "servicemap": { 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 3, 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:11:29.145470+0000", 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "services": { 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "osd": { 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "daemons": { 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "summary": "", 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "0": { 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "start_epoch": 0, 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "start_stamp": "0.000000", 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "gid": 0, 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "metadata": {}, 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "task_status": {} 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "2": { 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "start_epoch": 0, 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "start_stamp": "0.000000", 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "gid": 0, 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-24T18:11:31.551 INFO:tasks.workunit.client.0.trial172.stdout: "metadata": {}, 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: "task_status": {} 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: "mgrstat_first_committed": 1, 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: "mgrstat_last_committed": 36, 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: "logm_first_committed": 1, 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: "logm_last_committed": 210, 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: "paxos": { 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: "first_committed": 1, 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: "last_committed": 295, 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: "last_pn": 0, 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: "accepted_pn": 0 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:529: TEST_divergent_2: expr 2 + 2 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:529: TEST_divergent_2: objname=existing_4 2026-02-24T18:11:31.552 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:530: TEST_divergent_2: echo 'writing non-divergent object existing_4' 2026-02-24T18:11:31.553 INFO:tasks.workunit.client.0.trial172.stdout:writing non-divergent object existing_4 2026-02-24T18:11:31.553 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:531: TEST_divergent_2: ceph pg dump pgs 2026-02-24T18:11:31.735 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:11:31.736 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 100 0 0 51200 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:11:01.889289+0000 27'100 39:130 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:10:32.829947+0000 0'0 2026-02-24T18:10:32.829947+0000 0 0 periodic scrub scheduled @ 2026-02-26T00:54:27.887120+0000 0 0 2026-02-24T18:11:31.736 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:11:31.736 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:11:31.736 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:11:31.745 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:532: TEST_divergent_2: rados -p test put existing_4 /tmp/tmp.OgRbPGtOtd 2026-02-24T18:11:31.776 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:533: TEST_divergent_2: rm -f /tmp/tmp.OgRbPGtOtd 2026-02-24T18:11:31.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:535: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-24T18:11:31.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:535: TEST_divergent_2: wait_for_clean 2026-02-24T18:11:31.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:11:31.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:11:31.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:11:31.778 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-24T18:11:31.778 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:11:31.778 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:11:31.779 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:11:31.779 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:11:31.779 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:11:31.848 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-24T18:11:31.848 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:11:31.848 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:11:31.848 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:11:31.849 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:11:31.849 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:11:32.077 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:11:32.077 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:11:32.077 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:11:32.077 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:11:32.077 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:11:32.077 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:11:32.157 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=146028888075 2026-02-24T18:11:32.157 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 146028888075 2026-02-24T18:11:32.157 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-146028888075' 2026-02-24T18:11:32.157 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:11:32.157 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:11:32.222 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq= 2026-02-24T18:11:32.222 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: true 2026-02-24T18:11:32.222 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z '' 2026-02-24T18:11:32.222 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: flush_pg_stats: continue 2026-02-24T18:11:32.223 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:11:32.223 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:11:32.303 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=154618822667 2026-02-24T18:11:32.303 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 154618822667 2026-02-24T18:11:32.303 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-146028888075 2-154618822667' 2026-02-24T18:11:32.303 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:11:32.303 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-146028888075 2026-02-24T18:11:32.304 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:11:32.305 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:11:32.305 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-146028888075 2026-02-24T18:11:32.305 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:11:32.307 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=146028888075 2026-02-24T18:11:32.307 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 146028888075' 2026-02-24T18:11:32.307 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 146028888075 2026-02-24T18:11:32.307 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:11:32.538 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 146028888073 -lt 146028888075 2026-02-24T18:11:32.538 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:11:33.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:11:33.539 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:11:33.758 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 146028888075 -lt 146028888075 2026-02-24T18:11:33.759 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:11:33.759 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-154618822667 2026-02-24T18:11:33.759 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:11:33.760 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:11:33.761 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-154618822667 2026-02-24T18:11:33.761 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:11:33.762 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=154618822667 2026-02-24T18:11:33.762 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 154618822667' 2026-02-24T18:11:33.763 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 154618822667 2026-02-24T18:11:33.763 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:11:33.987 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 154618822667 -lt 154618822667 2026-02-24T18:11:33.988 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:11:33.988 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:33.988 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:34.268 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:11:34.268 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:34.268 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:34.269 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:34.269 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:34.269 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:34.269 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:34.269 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:34.477 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:34.477 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:34.478 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:34.478 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:34.768 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:34.768 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-24T18:11:34.768 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-24T18:11:34.768 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-24T18:11:34.768 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:34.768 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-24T18:11:34.870 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:11:34.870 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:34.871 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:34.871 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:34.871 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:34.871 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:34.872 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:34.872 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:35.090 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:35.090 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:35.091 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:35.091 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:35.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:35.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:11:35.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:11:35.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:11:35.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:11:35.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:11:35.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:11:35.383 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:11:35.383 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:11:35.678 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:11:35.678 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:11:35.678 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 8 )) 2026-02-24T18:11:35.678 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:35.678 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-24T18:11:35.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:11:35.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:35.881 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:35.881 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:35.881 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:35.881 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:35.881 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:35.882 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:36.096 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:36.097 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:36.098 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:36.098 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:36.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:36.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:11:36.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:11:36.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:11:36.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:11:36.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:11:36.382 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:11:36.382 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:11:36.382 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:11:36.674 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:11:36.675 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:11:36.675 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 8 )) 2026-02-24T18:11:36.675 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:36.675 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-24T18:11:37.076 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:11:37.076 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:37.076 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:37.076 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:37.076 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:37.077 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:37.077 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:37.077 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:37.297 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:37.298 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:37.298 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:37.298 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:37.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:37.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:11:37.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:11:37.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:11:37.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:11:37.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:11:37.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:11:37.592 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:11:37.592 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:11:37.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:11:37.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:11:37.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 8 )) 2026-02-24T18:11:37.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:37.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-24T18:11:38.677 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:11:38.677 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:38.678 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:38.678 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:38.678 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:38.679 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:38.679 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:38.679 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:38.892 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:38.892 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:38.893 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:38.893 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:39.180 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:39.180 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:11:39.180 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:11:39.180 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:11:39.180 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:11:39.180 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:11:39.180 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:11:39.180 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:11:39.180 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:11:39.469 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:11:39.469 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:11:39.469 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 8 )) 2026-02-24T18:11:39.469 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:39.469 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 1.6 2026-02-24T18:11:41.071 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:11:41.071 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:41.071 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:41.071 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:41.071 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:41.071 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:41.072 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:41.072 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:41.290 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:41.291 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:41.291 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:41.292 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:41.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:41.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:11:41.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:11:41.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:11:41.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:11:41.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:11:41.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:11:41.583 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:11:41.583 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:11:41.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:11:41.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:11:41.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 5 >= 8 )) 2026-02-24T18:11:41.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:41.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 3.2 2026-02-24T18:11:45.080 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:11:45.080 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:45.080 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:45.080 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:45.080 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:45.080 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:45.081 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:45.081 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:45.303 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:45.304 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:45.304 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:45.304 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:45.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:45.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:11:45.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:11:45.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:11:45.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:11:45.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:11:45.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:11:45.603 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:11:45.603 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:11:45.902 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:11:45.902 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:11:45.902 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 6 >= 8 )) 2026-02-24T18:11:45.902 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:45.902 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 6.4 2026-02-24T18:11:52.302 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:11:52.302 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:11:52.302 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:11:52.303 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:11:52.303 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:11:52.303 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:11:52.303 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:11:52.303 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:11:52.519 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:11:52.520 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:11:52.520 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:11:52.520 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:11:52.806 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:11:52.806 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:11:52.806 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:11:52.806 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:11:52.806 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:11:52.806 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:11:52.806 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:11:52.807 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:11:52.807 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:11:53.093 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:11:53.093 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:11:53.093 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 7 >= 8 )) 2026-02-24T18:11:53.093 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:11:53.093 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 7.3 2026-02-24T18:12:00.393 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:12:00.393 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:12:00.394 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:12:00.394 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:12:00.394 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:12:00.394 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:12:00.394 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:12:00.394 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:12:00.614 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:12:00.615 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:12:00.615 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:12:00.615 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:12:00.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:12:00.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:12:00.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:12:00.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:12:00.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:12:00.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:12:00.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:12:00.898 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:12:00.898 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:12:01.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:12:01.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:12:01.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 8 >= 8 )) 2026-02-24T18:12:01.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_fingerprint": "e511c6b5-e915-4e2d-bfe4-1d7c43be13b7", 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: "version": "20.3.0-5507-g618c318a", 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: "commit": "618c318a7a0b287ab5aeff207b7bf1d73ef45044", 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: "timestamp": "2026-02-24T18:12:01.459790+0000", 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: "tag": "", 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: "health": { 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: "status": "HEALTH_WARN", 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: "checks": { 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: "OSDMAP_FLAGS": { 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: "count": 17 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [], 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:12:01.469 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "OSD_DOWN": { 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "message": "1 osds down", 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "count": 1 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [ 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "message": "osd.1 (root=default,host=trial172) is down" 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "PG_DEGRADED": { 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "count": 1 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [ 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: "mutes": [] 2026-02-24T18:12:01.470 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "monmap_first_committed": 1, 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "monmap_last_committed": 1, 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "monmap": { 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 1, 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "fsid": "27512777-e37f-4f16-b176-1621842bd4df", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:10:20.627140Z", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "created": "2026-02-24T18:10:20.627140Z", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "min_mon_release": 20, 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "min_mon_release_name": "tentacle", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "election_strategy": 1, 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "disallowed_leaders": "", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode": false, 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "tiebreaker_mon": "", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "removed_ranks": "", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "features": { 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "persistent": [ 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "kraken", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "luminous", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "mimic", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap-prune", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "nautilus", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "octopus", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "pacific", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "elector-pinging", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "quincy", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "reef", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "squid", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "tentacle", 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: "nvmeof_beacon_diff" 2026-02-24T18:12:01.471 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "optional": [] 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "mons": [ 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "rank": 0, 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "name": "a", 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:7115", 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 0 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:7115/0", 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "priority": 0, 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 0, 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "time_added": "2026-02-24T18:10:20:627138+0000", 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "crush_location": [] 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "quorum": [ 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: 0 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap": { 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 39, 2026-02-24T18:12:01.472 INFO:tasks.workunit.client.0.trial172.stdout: "fsid": "27512777-e37f-4f16-b176-1621842bd4df", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "created": "2026-02-24T18:10:20.710665+0000", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:11:01.886611+0000", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "last_up_change": "2026-02-24T18:11:00.876859+0000", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "last_in_change": "2026-02-24T18:11:01.785925+0000", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "flags_num": 5800384, 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "flags_set": [ 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "nodown", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "noin", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "noout", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "pglog_hardlimit", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snapdirs", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "recovery_deletes", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "sortbitwise" 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "crush_version": 9, 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "full_ratio": 0.99000000953674316, 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_snapshot": "", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "pool_max": 1, 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "max_osd": 3, 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "require_min_compat_client": "luminous", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "min_compat_client": "jewel", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "require_osd_release": "tentacle", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "allow_crimson": false, 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "pools": [ 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "pool": 1, 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "pool_name": "test", 2026-02-24T18:12:01.473 INFO:tasks.workunit.client.0.trial172.stdout: "create_time": "2026-02-24T18:10:32.804007+0000", 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "flags": 1, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "flags_names": "hashpspool", 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "type": 1, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "size": 3, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "min_size": 2, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "crush_rule": 0, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_count": 0, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_target": 0, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_barrier": 0, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "is_stretch_pool": false, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "object_hash": 2, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "pg_autoscale_mode": "off", 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num": 1, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "pg_placement_num": 1, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "pg_placement_num_target": 1, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num_target": 1, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num_pending": 1, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "last_pg_merge_meta": { 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "source_pgid": "0.0", 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "ready_epoch": 0, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_started": 0, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_clean": 0, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "source_version": "0'0", 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "target_version": "0'0" 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "last_change": "27", 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend": "0", 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-24T18:12:01.474 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend_preluminous": "0", 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "auid": 0, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "snap_mode": "selfmanaged", 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "snap_seq": 0, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "snap_epoch": 0, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "pool_snaps": [], 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "removed_snaps": "[]", 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "quota_max_bytes": 0, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "quota_max_objects": 0, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "tiers": [], 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "tier_of": -1, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "read_tier": -1, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "write_tier": -1, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "cache_mode": "none", 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "target_max_bytes": 0, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "target_max_objects": 0, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "cache_min_flush_age": 0, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "cache_min_evict_age": 0, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "erasure_code_profile": "", 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_params": { 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "type": "none" 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_period": 0, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_count": 0, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "use_gmt_hitset": true, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "min_read_recency_for_promote": 0, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "min_write_recency_for_promote": 0, 2026-02-24T18:12:01.475 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_grade_decay_rate": 0, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_search_last_n": 0, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "grade_table": [], 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "stripe_width": 0, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "expected_num_objects": 0, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "fast_read": false, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "nonprimary_shards": "{}", 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "options": {}, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "application_metadata": {}, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "read_balance": { 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "score_type": "Fair distribution", 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "score_acting": 2.9900000095367432, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "score_stable": 2.9900000095367432, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "optimal_score": 0.67000001668930054, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "raw_score_acting": 2, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "raw_score_stable": 2, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity_weighted": 1, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "average_primary_affinity": 1, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "average_primary_affinity_weighted": 1 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "osds": [ 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "0d39d29d-6e3b-4fa3-a91e-35af674e24c9", 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "up": 1, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 5, 2026-02-24T18:12:01.476 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 27, 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 34, 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 38, 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 28, 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6800", 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6801", 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6802", 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6803", 2026-02-24T18:12:01.477 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6806", 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6807", 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6804", 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6805", 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 1761963574 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.478 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6801/1761963574", 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6803/1761963574", 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6807/1761963574", 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6805/1761963574", 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "exists", 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "up" 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "d7cb9f1a-c825-4426-aded-3118e4460724", 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "up": 0, 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 0, 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 0, 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 10, 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 21, 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 29, 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6810", 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.479 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6811", 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6812", 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6813", 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6816", 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:12:01.480 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6817", 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6814", 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6815", 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3631029146 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6811/3631029146", 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6813/3631029146", 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3631029146", 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3631029146", 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "exists" 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "aae44b3b-9b8d-43f3-925b-7313233d9556", 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "up": 1, 2026-02-24T18:12:01.481 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 14, 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 27, 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 36, 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 0, 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 30, 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6810", 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6811", 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6812", 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.482 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6813", 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6816", 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6817", 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6814", 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6815", 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3266666167 2026-02-24T18:12:01.483 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6811/3266666167", 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6813/3266666167", 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3266666167", 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3266666167", 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "exists", 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "up" 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "osd_xinfo": [ 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:10:56.483449+0000", 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:10:24.418099+0000", 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.484 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:12:01.485 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:10:56.774679+0000", 2026-02-24T18:12:01.485 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:12:01.485 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:12:01.485 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:12:01.485 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:12:01.485 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-24T18:12:01.485 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:12:01.485 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.485 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.485 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:10:57.078386+0000", 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:10:30.952922+0000", 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap": [], 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap_items": [], 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap_primaries": [], 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "pg_temp": [], 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "primary_temp": [], 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "blocklist": {}, 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "range_blocklist": {}, 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "erasure_code_profiles": { 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "default": { 2026-02-24T18:12:01.486 INFO:tasks.workunit.client.0.trial172.stdout: "k": "2", 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "m": "2", 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "plugin": "isa", 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "technique": "reed_sol_van" 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "removed_snaps_queue": [], 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "new_removed_snaps": [], 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "new_purged_snaps": [], 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "crush_node_flags": {}, 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "device_class_flags": {}, 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode": { 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode_enabled": false, 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_bucket_count": 0, 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "degraded_stretch_mode": 0, 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "recovering_stretch_mode": 0, 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode_bucket": 0 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "osd_metadata": [ 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.487 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6802/1761963574,v1:127.0.0.1:6803/1761963574]", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:12:01.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:10:21.741311Z", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6800/1761963574,v1:127.0.0.1:6801/1761963574]", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/1761963574,v1:127.0.0.1:6807/1761963574]", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/1761963574,v1:127.0.0.1:6805/1761963574]", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:12:01.489 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/0", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6812/3631029146,v1:127.0.0.1:6813/3631029146]", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:12:01.490 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:10:25.552571Z", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:12:01.491 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6810/3631029146,v1:127.0.0.1:6811/3631029146]", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3631029146,v1:127.0.0.1:6817/3631029146]", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3631029146,v1:127.0.0.1:6815/3631029146]", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/1", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6812/3266666167,v1:127.0.0.1:6813/3266666167]", 2026-02-24T18:12:01.492 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:12:01.493 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:10:28.432213Z", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6810/3266666167,v1:127.0.0.1:6811/3266666167]", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3266666167,v1:127.0.0.1:6817/3266666167]", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3266666167,v1:127.0.0.1:6815/3266666167]", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/2", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:12:01.494 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_clean_epochs": { 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "min_last_epoch_clean": 0, 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_clean": { 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "per_pool": [] 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "osd_epochs": [ 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 34 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 37 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_first_committed": 1, 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_last_committed": 39, 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "crushmap": { 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "devices": [ 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.0", 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:12:01.495 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.1", 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.2", 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: "types": [ 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 0, 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd" 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: "name": "host" 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 2, 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: "name": "chassis" 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 3, 2026-02-24T18:12:01.496 INFO:tasks.workunit.client.0.trial172.stdout: "name": "rack" 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 4, 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: "name": "row" 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 5, 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: "name": "pdu" 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 6, 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: "name": "pod" 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 7, 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: "name": "room" 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 8, 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: "name": "datacenter" 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 9, 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: "name": "zone" 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 10, 2026-02-24T18:12:01.497 INFO:tasks.workunit.client.0.trial172.stdout: "name": "region" 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "name": "root" 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "buckets": [ 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "id": -1, 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "name": "default", 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "root", 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "id": -3, 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "id": -2, 2026-02-24T18:12:01.498 INFO:tasks.workunit.client.0.trial172.stdout: "name": "default~ssd", 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "root", 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "id": -4, 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "id": -3, 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "name": "trial172", 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "host", 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:12:01.499 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 1 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 2 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "id": -4, 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "name": "trial172~ssd", 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "host", 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:12:01.500 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 1 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 2 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "rules": [ 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "rule_id": 0, 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "rule_name": "replicated_rule", 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "type": 1, 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "steps": [ 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "op": "take", 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "item": -1, 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "item_name": "default" 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "op": "choose_firstn", 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "num": 0, 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "type": "osd" 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: "op": "emit" 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:12:01.501 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: "tunables": { 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: "choose_local_tries": 0, 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: "choose_local_fallback_tries": 0, 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: "choose_total_tries": 50, 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_descend_once": 1, 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_vary_r": 1, 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_stable": 1, 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: "msr_descents": 100, 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: "msr_collision_tries": 100, 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: "straw_calc_version": 1, 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: "allowed_bucket_algs": 54, 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stderr:report 1820070009 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: "profile": "jewel", 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: "optimal_tunables": 1, 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: "legacy_tunables": 0, 2026-02-24T18:12:01.502 INFO:tasks.workunit.client.0.trial172.stdout: "minimum_required_version": "jewel", 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables": 1, 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables2": 1, 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "has_v2_rules": 0, 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables3": 1, 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "has_v3_rules": 0, 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "has_v4_buckets": 1, 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables5": 1, 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "has_v5_rules": 0, 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "has_msr_rules": 0 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "choose_args": {} 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "fsmap": { 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 1, 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "btime": "2026-02-24T18:10:20:710433+0000", 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "default_fscid": -1, 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "compat": { 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "compat": {}, 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "ro_compat": {}, 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "incompat": { 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_1": "base v0.20", 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_2": "client writeable ranges", 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_3": "default file layouts on dirs", 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_4": "dir inode in separate object", 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_5": "mds uses versioned encoding", 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_8": "no anchor table", 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_9": "file layout v2", 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_10": "snaprealm v2", 2026-02-24T18:12:01.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_11": "minor log segments", 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "feature_12": "quiesce subvolumes" 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "feature_flags": { 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "enable_multiple": true, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "ever_enabled_multiple": true 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "standbys": [], 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "filesystems": [] 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "mdsmap_first_committed": 1, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "mdsmap_last_committed": 1, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "auth": { 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "first_committed": 1, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "last_committed": 5, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_secrets": 3 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg": 1, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_active": 1, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_unknown": 0, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_osd": 3, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "pool_sum": { 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "stat_sum": { 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes": 51688, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects": 100, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_clones": 0, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_copies": 300, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing": 0, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_degraded": 100, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_misplaced": 0, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_unfound": 0, 2026-02-24T18:12:01.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_dirty": 100, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_whiteouts": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_read": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_read_kb": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_write": 101, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_write_kb": 101, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_scrub_errors": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_deep_scrub_errors": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_recovered": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_recovered": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_keys_recovered": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_omap": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_kb": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_kb": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_promote": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_high": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_low": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_some": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_full": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_pinned": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_legacy_snapsets": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_large_omap_objects": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_manifest": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_bytes": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_keys": 0, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_repaired": 0 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "store_stats": { 2026-02-24T18:12:01.505 INFO:tasks.workunit.client.0.trial172.stdout: "total": 0, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "available": 0, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 0, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 0, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 0, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 0 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "log_size": 17, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "ondisk_log_size": 17, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "up": 2, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "acting": 2, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "num_store_stats": 0 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "osd_sum": { 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 0, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 0, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "num_pgs": 3, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "num_osds": 3, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_osds": 3, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_omap_osds": 3, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "kb": 314572800, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used": 83840, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_data": 2056, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_omap": 24, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_meta": 81703, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "kb_avail": 314488960, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "statfs": { 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "total": 322122547200, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "available": 322036695040, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:12:01.506 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 2105344, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 415739, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 25401, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 83664071 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "hb_peers": [], 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "snap_trim_queue_len": 0, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "num_snap_trimming": 0, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "num_shards_repaired": 0, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "op_queue_age_hist": { 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "histogram": [], 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "upper_bound": 1 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "perf_stat": { 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ms": 0, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ms": 0, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ns": 0, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ns": 0 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "alerts": [], 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "network_ping_times": [] 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "osd_sum_by_class": { 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "ssd": { 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 0, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 0, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "num_pgs": 3, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "num_osds": 3, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_osds": 3, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_omap_osds": 3, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "kb": 314572800, 2026-02-24T18:12:01.507 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used": 83840, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_data": 2056, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_omap": 24, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_meta": 81703, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "kb_avail": 314488960, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "statfs": { 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "total": 322122547200, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "available": 322036695040, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 2105344, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 415739, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 25401, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 83664071 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "hb_peers": [], 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "snap_trim_queue_len": 0, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "num_snap_trimming": 0, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "num_shards_repaired": 0, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "op_queue_age_hist": { 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "histogram": [], 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "upper_bound": 1 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "perf_stat": { 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ms": 0, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ms": 0, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ns": 0, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ns": 0 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "alerts": [], 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "network_ping_times": [] 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: "pool_stats": [ 2026-02-24T18:12:01.508 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": 1, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg": 1, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "stat_sum": { 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes": 51688, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects": 100, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_clones": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_copies": 300, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_degraded": 100, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_misplaced": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_unfound": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_dirty": 100, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_whiteouts": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_read": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_read_kb": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_write": 101, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_write_kb": 101, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_scrub_errors": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_deep_scrub_errors": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_recovered": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_recovered": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_keys_recovered": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_omap": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_kb": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_kb": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_promote": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_high": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_low": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_some": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_full": 0, 2026-02-24T18:12:01.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_pinned": 0, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_legacy_snapsets": 0, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_large_omap_objects": 0, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_manifest": 0, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_bytes": 0, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_keys": 0, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_repaired": 0 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "store_stats": { 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "total": 0, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "available": 0, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 1220608, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 154528, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 0, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 0 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "log_size": 17, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "ondisk_log_size": 17, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "up": 2, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "acting": 2, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_store_stats": 3 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "osd_stats": [ 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 146028888081 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 42949672972 2026-02-24T18:12:01.510 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 154618822673 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_by_state": [ 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "state": "active+undersized+degraded", 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "num": 1 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "pool_pg_unavailable_map": [ 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": "1", 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "pgs": [] 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_by_osd": [ 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 1, 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 1, 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 1, 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 0, 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.511 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snaps": [ 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "pool": 1, 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snaps": {} 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "servicemap": { 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 5, 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:12:01.149334+0000", 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "services": {} 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "mgrstat_first_committed": 1, 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "mgrstat_last_committed": 51, 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "logm_first_committed": 1, 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "logm_last_committed": 272, 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "paxos": { 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "first_committed": 1, 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "last_committed": 372, 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "last_pn": 0, 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: "accepted_pn": 0 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stdout:delay recovery 2026-02-24T18:12:01.512 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:538: TEST_divergent_2: echo 'delay recovery' 2026-02-24T18:12:01.513 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:539: TEST_divergent_2: ceph pg dump pgs 2026-02-24T18:12:01.701 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:12:01.701 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 100 0 0 51688 0 0 17 84 17 active+undersized+degraded 2026-02-24T18:11:01.889289+0000 39'101 39:131 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:10:32.829947+0000 0'0 2026-02-24T18:10:32.829947+0000 0 0 periodic scrub scheduled @ 2026-02-26T00:54:27.887120+0000 0 0 2026-02-24T18:12:01.701 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:12:01.701 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:12:01.702 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:12:01.711 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: for i in $non_divergent 2026-02-24T18:12:01.712 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:542: TEST_divergent_2: get_asok_path osd.0 2026-02-24T18:12:01.712 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T18:12:01.712 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T18:12:01.712 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:12:01.712 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:01.712 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:01.713 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.0.asok 2026-02-24T18:12:01.713 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:542: TEST_divergent_2: CEPH_ARGS= 2026-02-24T18:12:01.713 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:542: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.0.asok set_recovery_delay 100000 2026-02-24T18:12:01.780 INFO:tasks.workunit.client.0.trial172.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-24T18:12:01.786 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: for i in $non_divergent 2026-02-24T18:12:01.786 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:542: TEST_divergent_2: get_asok_path osd.2 2026-02-24T18:12:01.786 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-24T18:12:01.787 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-24T18:12:01.787 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:12:01.787 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:01.787 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:01.787 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.2.asok 2026-02-24T18:12:01.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:542: TEST_divergent_2: CEPH_ARGS= 2026-02-24T18:12:01.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:542: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.2.asok set_recovery_delay 100000 2026-02-24T18:12:01.842 INFO:tasks.workunit.client.0.trial172.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-24T18:12:01.848 INFO:tasks.workunit.client.0.trial172.stdout:revive divergent 1 2026-02-24T18:12:01.848 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:546: TEST_divergent_2: echo 'revive divergent 1' 2026-02-24T18:12:01.848 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:547: TEST_divergent_2: ceph pg dump pgs 2026-02-24T18:12:02.070 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:12:02.071 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 100 0 0 51688 0 0 17 84 17 active+undersized+degraded 2026-02-24T18:11:01.889289+0000 39'101 39:131 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:10:32.829947+0000 0'0 2026-02-24T18:10:32.829947+0000 0 0 periodic scrub scheduled @ 2026-02-26T00:54:27.887120+0000 0 0 2026-02-24T18:12:02.071 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:12:02.071 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:12:02.071 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:12:02.080 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:548: TEST_divergent_2: ceph osd set noup 2026-02-24T18:12:02.409 INFO:tasks.workunit.client.0.trial172.stderr:noup is set 2026-02-24T18:12:02.421 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:549: TEST_divergent_2: activate_osd td/divergent-priors 1 2026-02-24T18:12:02.421 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:12:02.421 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:12:02.421 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T18:12:02.421 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:12:02.421 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-24T18:12:02.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:12:02.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:12:02.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:12:02.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:12:02.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-24T18:12:02.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-24T18:12:02.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:12:02.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:12:02.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:12:02.422 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:12:02.422 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:12:02.422 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:12:02.423 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:12:02.423 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:02.423 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:02.423 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:12:02.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:12:02.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:12:02.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:12:02.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:12:02.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:12:02.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:12:02.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:12:02.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:12:02.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:12:02.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:12:02.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-24T18:12:02.424 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:12:02.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T18:12:02.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:12:02.424 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-24T18:12:02.425 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T18:12:02.426 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:12:02.426 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:12:02.426 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:12:02.449 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:02.434+0000 7fe859673a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:02.450 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:02.438+0000 7fe859673a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:02.452 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:02.438+0000 7fe859673a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:02.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: return 0 2026-02-24T18:12:02.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:550: TEST_divergent_2: sleep 5 2026-02-24T18:12:02.851 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:02.838+0000 7fe859673a40 -1 Falling back to public interface 2026-02-24T18:12:03.567 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:03.554+0000 7fe859673a40 -1 osd.1 27 log_to_monitors true 2026-02-24T18:12:07.639 INFO:tasks.workunit.client.0.trial172.stdout:delay recovery divergent 2026-02-24T18:12:07.640 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:552: TEST_divergent_2: echo 'delay recovery divergent' 2026-02-24T18:12:07.640 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:553: TEST_divergent_2: ceph pg dump pgs 2026-02-24T18:12:07.839 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:12:07.839 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 100 0 0 51688 0 0 17 84 17 active+undersized+degraded 2026-02-24T18:11:01.889289+0000 39'101 42:136 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:10:32.829947+0000 0'0 2026-02-24T18:10:32.829947+0000 0 0 periodic scrub scheduled @ 2026-02-26T00:54:27.887120+0000 0 0 2026-02-24T18:12:07.839 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:12:07.839 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:12:07.840 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:12:07.850 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:554: TEST_divergent_2: get_asok_path osd.1 2026-02-24T18:12:07.851 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T18:12:07.851 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T18:12:07.851 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:12:07.851 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:07.851 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:07.851 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.1.asok 2026-02-24T18:12:07.852 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:554: TEST_divergent_2: CEPH_ARGS= 2026-02-24T18:12:07.852 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:554: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.1.asok set_recovery_delay 100000 2026-02-24T18:12:07.916 INFO:tasks.workunit.client.0.trial172.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-24T18:12:07.922 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:556: TEST_divergent_2: ceph osd unset noup 2026-02-24T18:12:08.251 INFO:tasks.workunit.client.0.trial172.stderr:noup is unset 2026-02-24T18:12:08.264 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:558: TEST_divergent_2: wait_for_osd up 0 2026-02-24T18:12:08.264 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:12:08.264 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:12:08.264 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:12:08.264 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:12:08.264 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:12:08.265 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:12:08.265 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:12:08.265 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:12:08.265 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:12:08.487 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 34 up_thru 38 down_at 28 last_clean_interval [5,27) [v2:127.0.0.1:6800/1761963574,v1:127.0.0.1:6801/1761963574] [v2:127.0.0.1:6802/1761963574,v1:127.0.0.1:6803/1761963574] exists,up 0d39d29d-6e3b-4fa3-a91e-35af674e24c9 2026-02-24T18:12:08.487 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:12:08.487 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:12:08.488 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:12:08.488 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:559: TEST_divergent_2: wait_for_osd up 1 2026-02-24T18:12:08.488 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:12:08.488 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:12:08.488 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:12:08.488 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:12:08.488 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:12:08.488 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:12:08.488 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:12:08.488 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:12:08.488 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:12:08.707 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 44 up_thru 44 down_at 29 last_clean_interval [10,27) [v2:127.0.0.1:6818/1756947405,v1:127.0.0.1:6819/1756947405] [v2:127.0.0.1:6820/1756947405,v1:127.0.0.1:6821/1756947405] exists,up d7cb9f1a-c825-4426-aded-3118e4460724 2026-02-24T18:12:08.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:12:08.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:12:08.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:12:08.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:560: TEST_divergent_2: wait_for_osd up 2 2026-02-24T18:12:08.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:12:08.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:12:08.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:12:08.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:12:08.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:12:08.710 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:12:08.710 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:12:08.710 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:12:08.711 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:12:08.927 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 36 up_thru 0 down_at 30 last_clean_interval [14,27) [v2:127.0.0.1:6810/3266666167,v1:127.0.0.1:6811/3266666167] [v2:127.0.0.1:6812/3266666167,v1:127.0.0.1:6813/3266666167] exists,up aae44b3b-9b8d-43f3-925b-7313233d9556 2026-02-24T18:12:08.927 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:12:08.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:12:08.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:12:08.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:562: TEST_divergent_2: ceph pg dump pgs 2026-02-24T18:12:09.133 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:12:09.133 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 100 0 0 51688 0 0 17 84 17 active+undersized+degraded 2026-02-24T18:11:01.889289+0000 39'101 42:136 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:10:32.829947+0000 0'0 2026-02-24T18:10:32.829947+0000 0 0 periodic scrub scheduled @ 2026-02-26T00:54:27.887120+0000 0 0 2026-02-24T18:12:09.133 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:12:09.133 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:12:09.133 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:12:09.142 INFO:tasks.workunit.client.0.trial172.stdout:wait for peering 2026-02-24T18:12:09.142 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:563: TEST_divergent_2: echo 'wait for peering' 2026-02-24T18:12:09.142 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:564: TEST_divergent_2: ceph pg dump pgs 2026-02-24T18:12:09.366 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:12:09.366 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 4 4 0 0 51688 0 0 11 90 11 active+recovering+degraded 2026-02-24T18:12:08.348753+0000 39'101 45:139 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:10:32.829947+0000 0'0 2026-02-24T18:10:32.829947+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:12:09.366 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:12:09.366 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:12:09.366 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:12:09.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:565: TEST_divergent_2: rados -p test put foo /tmp/tmp.AohTN555hP 2026-02-24T18:12:09.405 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:566: TEST_divergent_2: rm -f /tmp/tmp.AohTN555hP 2026-02-24T18:12:09.406 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:570: TEST_divergent_2: echo 'killing divergent 1' 2026-02-24T18:12:09.406 INFO:tasks.workunit.client.0.trial172.stdout:killing divergent 1 2026-02-24T18:12:09.406 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:571: TEST_divergent_2: ceph pg dump pgs 2026-02-24T18:12:09.623 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:12:09.623 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 4 4 0 0 51688 0 0 11 90 11 active+recovering+degraded 2026-02-24T18:12:08.348753+0000 39'101 45:139 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:10:32.829947+0000 0'0 2026-02-24T18:10:32.829947+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:12:09.623 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:12:09.623 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:12:09.623 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:12:09.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:572: TEST_divergent_2: kill_daemons td/divergent-priors KILL osd.1 2026-02-24T18:12:09.632 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:12:09.632 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:12:09.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:12:09.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:12:09.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:12:09.741 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:12:09.741 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:575: TEST_divergent_2: expfile=td/divergent-priors/exp.19163.out 2026-02-24T18:12:09.741 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:576: TEST_divergent_2: _objectstore_tool_nodown td/divergent-priors 1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.19163.out 2026-02-24T18:12:09.741 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-24T18:12:09.742 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T18:12:09.742 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-24T18:12:09.742 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T18:12:09.742 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-24T18:12:09.742 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.19163.out 2026-02-24T18:12:09.930 INFO:tasks.workunit.client.0.trial172.stderr:Exporting 1.0 info 1.0( v 46'102 lc 0'0 (27'90,46'102] local-lis/les=44/45 n=101 ec=21/21 lis/c=44/21 les/c/f=45/22/0 sis=44) 2026-02-24T18:12:09.950 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:023dd3d2:::existing_12:head# 2026-02-24T18:12:09.950 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:04c40a0c:::existing_93:head# 2026-02-24T18:12:09.950 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:064ff5fb:::existing_36:head# 2026-02-24T18:12:09.950 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:091f9f1d:::existing_94:head# 2026-02-24T18:12:09.950 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:0a865a93:::existing_54:head# 2026-02-24T18:12:09.950 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:0fed513c:::existing_41:head# 2026-02-24T18:12:09.950 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:13b9e0b8:::existing_92:head# 2026-02-24T18:12:09.950 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:169157fa:::existing_81:head# 2026-02-24T18:12:09.950 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:17731382:::existing_58:head# 2026-02-24T18:12:09.951 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:18a45a39:::existing_43:head# 2026-02-24T18:12:09.951 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:1c160f01:::existing_66:head# 2026-02-24T18:12:09.951 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:1dd6c249:::existing_97:head# 2026-02-24T18:12:09.951 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:227e0c77:::existing_7:head# 2026-02-24T18:12:09.951 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:23e1b063:::existing_59:head# 2026-02-24T18:12:09.951 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:28566edb:::existing_78:head# 2026-02-24T18:12:09.951 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:2a101182:::existing_20:head# 2026-02-24T18:12:09.951 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:2d1519da:::existing_96:head# 2026-02-24T18:12:09.951 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:3078b87a:::existing_86:head# 2026-02-24T18:12:09.951 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:30bde01b:::existing_15:head# 2026-02-24T18:12:09.951 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:317a7e19:::existing_56:head# 2026-02-24T18:12:09.951 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:38b9c7fa:::existing_33:head# 2026-02-24T18:12:09.951 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:3e16d84b:::existing_48:head# 2026-02-24T18:12:09.952 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:45fe8203:::existing_75:head# 2026-02-24T18:12:09.952 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:4e82c9ba:::existing_50:head# 2026-02-24T18:12:09.952 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:5298dcdb:::existing_57:head# 2026-02-24T18:12:09.952 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:52ebaa44:::existing_32:head# 2026-02-24T18:12:09.952 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:53588912:::existing_9:head# 2026-02-24T18:12:09.952 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:5b7e2de6:::existing_84:head# 2026-02-24T18:12:09.952 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:5da0255c:::existing_13:head# 2026-02-24T18:12:09.952 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:5da2dab8:::existing_53:head# 2026-02-24T18:12:09.952 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:5ee3514b:::existing_30:head# 2026-02-24T18:12:09.952 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:602f83fe:::foo:head# 2026-02-24T18:12:09.952 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:60426cc7:::existing_62:head# 2026-02-24T18:12:09.953 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:628918ca:::existing_79:head# 2026-02-24T18:12:09.953 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:63f8ab2d:::existing_77:head# 2026-02-24T18:12:09.953 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:67e1780f:::existing_27:head# 2026-02-24T18:12:09.953 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:68143ca1:::existing_71:head# 2026-02-24T18:12:09.953 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:6bac9509:::existing_29:head# 2026-02-24T18:12:09.953 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:6bbec798:::existing_45:head# 2026-02-24T18:12:09.953 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:6cb23a40:::existing_65:head# 2026-02-24T18:12:09.953 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:7055ec5e:::existing_83:head# 2026-02-24T18:12:09.953 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:70a54e2b:::existing_87:head# 2026-02-24T18:12:09.953 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:750ace21:::existing_19:head# 2026-02-24T18:12:09.953 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:78623b1d:::existing_6:head# 2026-02-24T18:12:09.953 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:79881152:::existing_63:head# 2026-02-24T18:12:09.953 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:7a80c0d5:::existing_99:head# 2026-02-24T18:12:09.954 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:81490677:::existing_76:head# 2026-02-24T18:12:09.954 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:85ec8e5d:::existing_85:head# 2026-02-24T18:12:09.954 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:8e2abbf0:::existing_49:head# 2026-02-24T18:12:09.954 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:8ee0fe0f:::existing_52:head# 2026-02-24T18:12:09.954 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:923c5d4d:::existing_5:head# 2026-02-24T18:12:09.954 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:92731e71:::existing_89:head# 2026-02-24T18:12:09.954 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:928205cf:::existing_82:head# 2026-02-24T18:12:09.954 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:93be2267:::existing_24:head# 2026-02-24T18:12:09.954 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:99ceda2c:::existing_44:head# 2026-02-24T18:12:09.954 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:9dd917ee:::existing_26:head# 2026-02-24T18:12:09.954 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:a28805a6:::existing_72:head# 2026-02-24T18:12:09.954 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:a294a8f6:::existing_16:head# 2026-02-24T18:12:09.954 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:a2f5473a:::existing_42:head# 2026-02-24T18:12:09.955 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:a504a533:::existing_90:head# 2026-02-24T18:12:09.955 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:a691337e:::existing_61:head# 2026-02-24T18:12:09.955 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:a6e8d7ac:::existing_28:head# 2026-02-24T18:12:09.955 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:a9fde86e:::existing_18:head# 2026-02-24T18:12:09.955 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:aa309faa:::existing_68:head# 2026-02-24T18:12:09.955 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:ae40b0ed:::existing_31:head# 2026-02-24T18:12:09.955 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:b279b1fe:::existing_10:head# 2026-02-24T18:12:09.955 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:b343e277:::existing_88:head# 2026-02-24T18:12:09.955 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:b7130e90:::existing_80:head# 2026-02-24T18:12:09.955 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:bc6f7309:::existing_70:head# 2026-02-24T18:12:09.955 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:be8489b7:::existing_38:head# 2026-02-24T18:12:09.955 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:beb4e5e3:::existing_47:head# 2026-02-24T18:12:09.955 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:bebec877:::existing_74:head# 2026-02-24T18:12:09.956 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:c04347d8:::existing_60:head# 2026-02-24T18:12:09.956 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:c156033e:::existing_100:head# 2026-02-24T18:12:09.956 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:c19f5a8e:::existing_8:head# 2026-02-24T18:12:09.956 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:c3e6ebe4:::existing_17:head# 2026-02-24T18:12:09.956 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:c6c5a7c2:::existing_39:head# 2026-02-24T18:12:09.956 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:cc1d0abf:::existing_95:head# 2026-02-24T18:12:09.956 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:ccb3865b:::existing_73:head# 2026-02-24T18:12:09.956 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:cd84e19e:::existing_40:head# 2026-02-24T18:12:09.956 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:d2cd64ce:::existing_37:head# 2026-02-24T18:12:09.956 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:d41a25c9:::existing_22:head# 2026-02-24T18:12:09.956 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:d7f088da:::existing_91:head# 2026-02-24T18:12:09.957 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:d9cbd1d6:::existing_64:head# 2026-02-24T18:12:09.957 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:dcc4ff82:::existing_98:head# 2026-02-24T18:12:09.957 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:dd677bd3:::existing_35:head# 2026-02-24T18:12:09.957 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:dd6d90e7:::existing_46:head# 2026-02-24T18:12:09.957 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:e0e22077:::existing_21:head# 2026-02-24T18:12:09.957 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:e255b710:::existing_55:head# 2026-02-24T18:12:09.957 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:eff1d5db:::existing_14:head# 2026-02-24T18:12:09.957 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:f1b156bf:::existing_23:head# 2026-02-24T18:12:09.957 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:f5ce4199:::existing_34:head# 2026-02-24T18:12:09.957 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:f6809456:::existing_11:head# 2026-02-24T18:12:09.957 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:f699a2a9:::existing_67:head# 2026-02-24T18:12:09.957 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:f9baac43:::existing_25:head# 2026-02-24T18:12:09.958 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:fb76688e:::existing_51:head# 2026-02-24T18:12:09.958 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:fbb92eec:::existing_69:head# 2026-02-24T18:12:09.958 INFO:tasks.workunit.client.0.trial172.stderr:Export successful 2026-02-24T18:12:09.958 INFO:tasks.workunit.client.0.trial172.stdout: marking collection for removal 2026-02-24T18:12:09.959 INFO:tasks.workunit.client.0.trial172.stdout:setting '_remove' omap key 2026-02-24T18:12:09.959 INFO:tasks.workunit.client.0.trial172.stdout:finish_remove_pgs 1.0_head removing 1.0 2026-02-24T18:12:09.959 INFO:tasks.workunit.client.0.trial172.stderr:Remove successful 2026-02-24T18:12:10.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:577: TEST_divergent_2: _objectstore_tool_nodown td/divergent-priors 1 --op import --file td/divergent-priors/exp.19163.out 2026-02-24T18:12:10.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-24T18:12:10.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T18:12:10.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-24T18:12:10.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T18:12:10.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-24T18:12:10.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op import --file td/divergent-priors/exp.19163.out 2026-02-24T18:12:10.245 INFO:tasks.workunit.client.0.trial172.stdout:get_pg_num_history pg_num_history pg_num_history(e46 pg_nums {1={21=1}} deleted_pools ) 2026-02-24T18:12:10.245 INFO:tasks.workunit.client.0.trial172.stdout:Importing pgid 1.0 2026-02-24T18:12:10.245 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:023dd3d2:::existing_12:head# 2026-02-24T18:12:10.245 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.245 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:04c40a0c:::existing_93:head# 2026-02-24T18:12:10.245 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.245 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:064ff5fb:::existing_36:head# 2026-02-24T18:12:10.246 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.246 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:091f9f1d:::existing_94:head# 2026-02-24T18:12:10.246 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.246 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:0a865a93:::existing_54:head# 2026-02-24T18:12:10.246 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.247 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:0fed513c:::existing_41:head# 2026-02-24T18:12:10.247 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.247 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:13b9e0b8:::existing_92:head# 2026-02-24T18:12:10.247 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.247 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:169157fa:::existing_81:head# 2026-02-24T18:12:10.247 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.247 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:17731382:::existing_58:head# 2026-02-24T18:12:10.247 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.247 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:18a45a39:::existing_43:head# 2026-02-24T18:12:10.247 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.247 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:1c160f01:::existing_66:head# 2026-02-24T18:12:10.247 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.247 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:1dd6c249:::existing_97:head# 2026-02-24T18:12:10.247 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:227e0c77:::existing_7:head# 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:23e1b063:::existing_59:head# 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:28566edb:::existing_78:head# 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:2a101182:::existing_20:head# 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:2d1519da:::existing_96:head# 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:3078b87a:::existing_86:head# 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:30bde01b:::existing_15:head# 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:317a7e19:::existing_56:head# 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:38b9c7fa:::existing_33:head# 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:3e16d84b:::existing_48:head# 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.248 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:45fe8203:::existing_75:head# 2026-02-24T18:12:10.249 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:4e82c9ba:::existing_50:head# 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:5298dcdb:::existing_57:head# 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:52ebaa44:::existing_32:head# 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:53588912:::existing_9:head# 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:5b7e2de6:::existing_84:head# 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:5da0255c:::existing_13:head# 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:5da2dab8:::existing_53:head# 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:5ee3514b:::existing_30:head# 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:602f83fe:::foo:head# 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:60426cc7:::existing_62:head# 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:628918ca:::existing_79:head# 2026-02-24T18:12:10.250 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.251 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:63f8ab2d:::existing_77:head# 2026-02-24T18:12:10.251 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.251 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:67e1780f:::existing_27:head# 2026-02-24T18:12:10.251 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.251 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:68143ca1:::existing_71:head# 2026-02-24T18:12:10.251 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:6bac9509:::existing_29:head# 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:6bbec798:::existing_45:head# 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:6cb23a40:::existing_65:head# 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:7055ec5e:::existing_83:head# 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:70a54e2b:::existing_87:head# 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:750ace21:::existing_19:head# 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:78623b1d:::existing_6:head# 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:79881152:::existing_63:head# 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.252 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:7a80c0d5:::existing_99:head# 2026-02-24T18:12:10.253 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.253 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:81490677:::existing_76:head# 2026-02-24T18:12:10.253 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.253 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:85ec8e5d:::existing_85:head# 2026-02-24T18:12:10.253 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.253 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:8e2abbf0:::existing_49:head# 2026-02-24T18:12:10.253 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.253 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:8ee0fe0f:::existing_52:head# 2026-02-24T18:12:10.253 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.253 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:923c5d4d:::existing_5:head# 2026-02-24T18:12:10.253 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.253 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:92731e71:::existing_89:head# 2026-02-24T18:12:10.253 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.253 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:928205cf:::existing_82:head# 2026-02-24T18:12:10.253 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.254 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:93be2267:::existing_24:head# 2026-02-24T18:12:10.254 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.254 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:99ceda2c:::existing_44:head# 2026-02-24T18:12:10.254 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.254 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:9dd917ee:::existing_26:head# 2026-02-24T18:12:10.254 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.254 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:a28805a6:::existing_72:head# 2026-02-24T18:12:10.254 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.254 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:a294a8f6:::existing_16:head# 2026-02-24T18:12:10.254 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.254 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:a2f5473a:::existing_42:head# 2026-02-24T18:12:10.254 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.254 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:a504a533:::existing_90:head# 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:a691337e:::existing_61:head# 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:a6e8d7ac:::existing_28:head# 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:a9fde86e:::existing_18:head# 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:aa309faa:::existing_68:head# 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:ae40b0ed:::existing_31:head# 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:b279b1fe:::existing_10:head# 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:b343e277:::existing_88:head# 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:b7130e90:::existing_80:head# 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:bc6f7309:::existing_70:head# 2026-02-24T18:12:10.255 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.256 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:be8489b7:::existing_38:head# 2026-02-24T18:12:10.256 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:beb4e5e3:::existing_47:head# 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:bebec877:::existing_74:head# 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:c04347d8:::existing_60:head# 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:c156033e:::existing_100:head# 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:c19f5a8e:::existing_8:head# 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:c3e6ebe4:::existing_17:head# 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:c6c5a7c2:::existing_39:head# 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:cc1d0abf:::existing_95:head# 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:ccb3865b:::existing_73:head# 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:cd84e19e:::existing_40:head# 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:d2cd64ce:::existing_37:head# 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:d41a25c9:::existing_22:head# 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:d7f088da:::existing_91:head# 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:d9cbd1d6:::existing_64:head# 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.257 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:dcc4ff82:::existing_98:head# 2026-02-24T18:12:10.258 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.258 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:dd677bd3:::existing_35:head# 2026-02-24T18:12:10.258 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.258 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:dd6d90e7:::existing_46:head# 2026-02-24T18:12:10.258 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:e0e22077:::existing_21:head# 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:e255b710:::existing_55:head# 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:eff1d5db:::existing_14:head# 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:f1b156bf:::existing_23:head# 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:f5ce4199:::existing_34:head# 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:f6809456:::existing_11:head# 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:f699a2a9:::existing_67:head# 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:f9baac43:::existing_25:head# 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:fb76688e:::existing_51:head# 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:fbb92eec:::existing_69:head# 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:write_pg epoch 46 info 1.0( v 46'102 lc 0'0 (27'90,46'102] local-lis/les=44/45 n=101 ec=21/21 lis/c=44/21 les/c/f=45/22/0 sis=44) 2026-02-24T18:12:10.259 INFO:tasks.workunit.client.0.trial172.stdout:Import successful 2026-02-24T18:12:10.366 INFO:tasks.workunit.client.0.trial172.stdout:reviving divergent 1 2026-02-24T18:12:10.366 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:579: TEST_divergent_2: echo 'reviving divergent 1' 2026-02-24T18:12:10.366 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:580: TEST_divergent_2: ceph pg dump pgs 2026-02-24T18:12:10.565 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:12:10.565 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 4 4 0 0 51688 0 0 11 90 11 active+recovering+degraded 2026-02-24T18:12:08.348753+0000 39'101 45:139 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:10:32.829947+0000 0'0 2026-02-24T18:10:32.829947+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:12:10.565 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:12:10.565 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:12:10.565 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:12:10.574 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:581: TEST_divergent_2: activate_osd td/divergent-priors 1 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:12:10.575 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:12:10.576 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:12:10.576 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:10.576 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:10.576 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:12:10.576 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:12:10.576 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:12:10.576 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:12:10.576 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:12:10.576 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:12:10.576 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:12:10.576 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:12:10.576 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:12:10.576 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:12:10.576 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:12:10.576 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-24T18:12:10.577 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:12:10.577 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T18:12:10.577 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:12:10.577 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-24T18:12:10.578 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T18:12:10.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:12:10.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:12:10.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:12:10.602 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:10.590+0000 7ff1cae87a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:10.603 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:10.590+0000 7ff1cae87a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:10.604 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:10.590+0000 7ff1cae87a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:10.804 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T18:12:10.805 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:12:10.805 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:12:10.805 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:12:10.805 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:12:10.805 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:12:10.805 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:12:10.805 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:12:10.805 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:12:10.805 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:12:11.018 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 44 up_thru 44 down_at 29 last_clean_interval [10,27) [v2:127.0.0.1:6818/1756947405,v1:127.0.0.1:6819/1756947405] [v2:127.0.0.1:6820/1756947405,v1:127.0.0.1:6821/1756947405] exists,up d7cb9f1a-c825-4426-aded-3118e4460724 2026-02-24T18:12:11.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:12:11.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:12:11.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:12:11.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:582: TEST_divergent_2: wait_for_osd up 1 2026-02-24T18:12:11.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:12:11.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:12:11.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:12:11.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:12:11.019 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:12:11.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:12:11.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:12:11.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:12:11.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:12:11.175 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:11.162+0000 7ff1cae87a40 -1 Falling back to public interface 2026-02-24T18:12:11.232 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 44 up_thru 44 down_at 29 last_clean_interval [10,27) [v2:127.0.0.1:6818/1756947405,v1:127.0.0.1:6819/1756947405] [v2:127.0.0.1:6820/1756947405,v1:127.0.0.1:6821/1756947405] exists,up d7cb9f1a-c825-4426-aded-3118e4460724 2026-02-24T18:12:11.233 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:12:11.233 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:12:11.233 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:12:11.233 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:584: TEST_divergent_2: sleep 20 2026-02-24T18:12:11.769 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:11.754+0000 7ff1cae87a40 -1 osd.1 46 log_to_monitors true 2026-02-24T18:12:31.234 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:585: TEST_divergent_2: get_asok_path osd.1 2026-02-24T18:12:31.235 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T18:12:31.235 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T18:12:31.236 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:12:31.236 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:31.236 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:31.236 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.1.asok 2026-02-24T18:12:31.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:585: TEST_divergent_2: CEPH_ARGS= 2026-02-24T18:12:31.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:585: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.1.asok dump_ops_in_flight 2026-02-24T18:12:31.300 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:12:31.300 INFO:tasks.workunit.client.0.trial172.stdout: "ops": [], 2026-02-24T18:12:31.300 INFO:tasks.workunit.client.0.trial172.stdout: "num_ops": 0 2026-02-24T18:12:31.300 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:12:31.305 INFO:tasks.workunit.client.0.trial172.stdout:allowing recovery 2026-02-24T18:12:31.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:587: TEST_divergent_2: echo 'allowing recovery' 2026-02-24T18:12:31.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:588: TEST_divergent_2: ceph pg dump pgs 2026-02-24T18:12:31.522 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:12:31.522 INFO:tasks.workunit.client.0.trial172.stdout:1.0 101 0 0 0 0 52200 0 0 12 90 12 active+clean 2026-02-24T18:12:12.601645+0000 46'102 50:164 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:10:32.829947+0000 0'0 2026-02-24T18:10:32.829947+0000 0 0 periodic scrub scheduled @ 2026-02-25T20:41:41.392350+0000 0 0 2026-02-24T18:12:31.522 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:12:31.522 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:12:31.523 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:12:31.532 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:590: TEST_divergent_2: for i in $osds 2026-02-24T18:12:31.533 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:592: TEST_divergent_2: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-24T18:12:31.606 INFO:tasks.workunit.client.0.trial172.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-24T18:12:31.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:590: TEST_divergent_2: for i in $osds 2026-02-24T18:12:31.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:592: TEST_divergent_2: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-24T18:12:31.690 INFO:tasks.workunit.client.0.trial172.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-24T18:12:31.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:590: TEST_divergent_2: for i in $osds 2026-02-24T18:12:31.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:592: TEST_divergent_2: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-24T18:12:31.778 INFO:tasks.workunit.client.0.trial172.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-24T18:12:31.785 INFO:tasks.workunit.client.0.trial172.stdout:reading divergent objects 2026-02-24T18:12:31.786 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:595: TEST_divergent_2: echo 'reading divergent objects' 2026-02-24T18:12:31.786 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: ceph pg dump pgs 2026-02-24T18:12:31.999 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:12:32.000 INFO:tasks.workunit.client.0.trial172.stdout:1.0 101 0 0 0 0 52200 0 0 12 90 12 active+clean 2026-02-24T18:12:12.601645+0000 46'102 50:164 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:10:32.829947+0000 0'0 2026-02-24T18:10:32.829947+0000 0 0 periodic scrub scheduled @ 2026-02-25T20:41:41.392350+0000 0 0 2026-02-24T18:12:32.000 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:12:32.000 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:12:32.000 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:12:32.011 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:597: TEST_divergent_2: expr 2 + 2 2026-02-24T18:12:32.012 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:597: TEST_divergent_2: seq 1 4 2026-02-24T18:12:32.013 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:597: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:12:32.013 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:599: TEST_divergent_2: rados -p test get existing_1 td/divergent-priors/existing 2026-02-24T18:12:32.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:597: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:12:32.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:599: TEST_divergent_2: rados -p test get existing_2 td/divergent-priors/existing 2026-02-24T18:12:32.064 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:597: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:12:32.064 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:599: TEST_divergent_2: rados -p test get existing_3 td/divergent-priors/existing 2026-02-24T18:12:32.092 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:597: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:12:32.092 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:599: TEST_divergent_2: rados -p test get existing_4 td/divergent-priors/existing 2026-02-24T18:12:32.121 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:601: TEST_divergent_2: seq 1 2 2026-02-24T18:12:32.122 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:601: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-24T18:12:32.123 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:603: TEST_divergent_2: rados -p test get newobject_1 td/divergent-priors/existing 2026-02-24T18:12:32.142 INFO:tasks.workunit.client.0.trial172.stderr:error getting test/newobject_1: (2) No such file or directory 2026-02-24T18:12:32.144 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:601: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-24T18:12:32.144 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:603: TEST_divergent_2: rados -p test get newobject_2 td/divergent-priors/existing 2026-02-24T18:12:32.170 INFO:tasks.workunit.client.0.trial172.stderr:error getting test/newobject_2: (2) No such file or directory 2026-02-24T18:12:32.172 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:605: TEST_divergent_2: rm -f td/divergent-priors/existing 2026-02-24T18:12:32.174 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:607: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2026-02-24T18:12:32.175 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:607: TEST_divergent_2: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 20 _merge_object_divergent_entries: merging hoid 1:70b3bf12:::existing_4:head entries: 27'106 (27'4) delete 1:70b3bf12:::existing_4:head by client.4545.0:1 2026-02-24T18:10:45.899139+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 20 _merge_object_divergent_entries: keeping 27'106 (27'4) delete 1:70b3bf12:::existing_4:head by client.4545.0:1 2026-02-24T18:10:45.899139+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:70b3bf12:::existing_4:head prior_version: 27'4 first_divergent_update: 27'106 last_divergent_update: 27'106 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: more recent entry found: 39'101 (27'4) modify 1:70b3bf12:::existing_4:head by client.4654.0:1 2026-02-24T18:11:31.761100+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false, already merged 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 20 _merge_object_divergent_entries: merging hoid 1:75ac5091:::newobject_2:head entries: 27'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4535.0:1 2026-02-24T18:10:45.893247+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 20 _merge_object_divergent_entries: keeping 27'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4535.0:1 2026-02-24T18:10:45.893247+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version: 0'0 first_divergent_update: 27'101 last_divergent_update: 27'101 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head has no more recent entries in log 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version or op type indicates creation, deleting 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 20 _merge_object_divergent_entries: merging hoid 1:8881e6d8:::existing_3:head entries: 27'105 (27'3) delete 1:8881e6d8:::existing_3:head by client.4543.0:1 2026-02-24T18:10:45.897488+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 20 _merge_object_divergent_entries: keeping 27'105 (27'3) delete 1:8881e6d8:::existing_3:head by client.4543.0:1 2026-02-24T18:10:45.897488+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid object_not_in_store: 1 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version: 27'3 first_divergent_update: 27'105 last_divergent_update: 27'105 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head has no more recent entries in log 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head must be rolled back or recovered, attempting to rollback 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head olog_can_rollback_to: 27'106 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot rollback 27'105 (27'3) delete 1:8881e6d8:::existing_3:head by client.4543.0:1 2026-02-24T18:10:45.897488+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot roll back, removing and adding to missing 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version 27'3 <= info.log_tail 27'90 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 20 _merge_object_divergent_entries: merging hoid 1:97db2a0f:::existing_2:head entries: 27'104 (27'2) modify 1:97db2a0f:::existing_2:head by client.4539.0:1 2026-02-24T18:10:45.895147+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:12:32.178 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 20 _merge_object_divergent_entries: keeping 27'104 (27'2) modify 1:97db2a0f:::existing_2:head by client.4539.0:1 2026-02-24T18:10:45.895147+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version: 27'2 first_divergent_update: 27'104 last_divergent_update: 27'104 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head has no more recent entries in log 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head must be rolled back or recovered, attempting to rollback 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head olog_can_rollback_to: 27'106 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot rollback 27'104 (27'2) modify 1:97db2a0f:::existing_2:head by client.4539.0:1 2026-02-24T18:10:45.895147+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot roll back, removing and adding to missing 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version 27'2 <= info.log_tail 27'90 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 20 _merge_object_divergent_entries: merging hoid 1:d858c035:::existing_1:head entries: 27'103 (27'1) modify 1:d858c035:::existing_1:head by client.4536.0:1 2026-02-24T18:10:45.894079+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 20 _merge_object_divergent_entries: keeping 27'103 (27'1) modify 1:d858c035:::existing_1:head by client.4536.0:1 2026-02-24T18:10:45.894079+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version: 27'1 first_divergent_update: 27'103 last_divergent_update: 27'103 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head has no more recent entries in log 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head must be rolled back or recovered, attempting to rollback 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head olog_can_rollback_to: 27'106 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot rollback 27'103 (27'1) modify 1:d858c035:::existing_1:head by client.4536.0:1 2026-02-24T18:10:45.894079+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot roll back, removing and adding to missing 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version 27'1 <= info.log_tail 27'90 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 20 _merge_object_divergent_entries: merging hoid 1:da59f6e7:::newobject_1:head entries: 27'102 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4537.0:1 2026-02-24T18:10:45.893871+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-24T18:12:32.179 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 20 _merge_object_divergent_entries: keeping 27'102 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4537.0:1 2026-02-24T18:10:45.893871+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-24T18:12:32.180 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-24T18:12:32.180 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version: 0'0 first_divergent_update: 27'102 last_divergent_update: 27'102 2026-02-24T18:12:32.180 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head has no more recent entries in log 2026-02-24T18:12:32.180 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:12:08.274+0000 7fe83b5f9640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version or op type indicates creation, deleting 2026-02-24T18:12:32.180 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:609: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2026-02-24T18:12:32.181 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:609: TEST_divergent_2: grep -q '_merge_object_divergent_entries: more recent entry found:' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-24T18:12:32.185 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:615: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2026-02-24T18:12:32.186 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:615: TEST_divergent_2: grep -q '_merge_object_divergent_entries.*prior_version or op type indicates creation' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-24T18:12:32.189 INFO:tasks.workunit.client.0.trial172.stdout:success 2026-02-24T18:12:32.189 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:620: TEST_divergent_2: echo success 2026-02-24T18:12:32.189 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:622: TEST_divergent_2: rm td/divergent-priors/td/divergent-priors/exp.19163.out 2026-02-24T18:12:32.190 INFO:tasks.workunit.client.0.trial172.stderr:rm: cannot remove 'td/divergent-priors/td/divergent-priors/exp.19163.out': No such file or directory 2026-02-24T18:12:32.190 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:623: TEST_divergent_2: delete_pool test 2026-02-24T18:12:32.190 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=test 2026-02-24T18:12:32.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-24T18:12:32.454 INFO:tasks.workunit.client.0.trial172.stderr:pool 'test' does not exist 2026-02-24T18:12:32.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:624: TEST_divergent_2: kill_daemons td/divergent-priors 2026-02-24T18:12:32.465 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:12:32.465 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:12:32.465 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:12:32.466 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:12:32.466 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:12:37.780 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:12:37.780 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-24T18:12:37.780 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-24T18:12:37.780 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:12:37.780 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-24T18:12:37.780 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:12:37.780 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:12:37.780 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:12:37.781 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:12:37.781 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:12:37.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:12:37.788 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:12:37.789 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:12:37.789 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:12:37.791 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:12:37.791 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:12:37.791 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:12:37.792 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:12:37.792 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:12:37.793 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:12:37.793 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:12:37.794 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:12:37.795 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:12:37.795 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-24T18:12:37.816 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:12:37.816 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:37.816 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:37.816 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19163 2026-02-24T18:12:37.818 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:12:37.818 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:12:37.818 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-24T18:12:37.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-24T18:12:37.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/divergent-priors 2026-02-24T18:12:37.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/divergent-priors 2026-02-24T18:12:37.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-24T18:12:37.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:12:37.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-24T18:12:37.819 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:12:37.819 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:12:37.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:12:37.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:12:37.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:12:37.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:12:37.821 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:12:37.821 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:12:37.822 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:12:37.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:12:37.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:12:37.823 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:12:37.824 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:12:37.824 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:12:37.825 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:12:37.825 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:12:37.826 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:12:37.827 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:12:37.827 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-24T18:12:37.828 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:12:37.829 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:37.829 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:37.829 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19163 2026-02-24T18:12:37.830 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:12:37.830 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:12:37.830 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/divergent-priors 2026-02-24T18:12:37.831 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T18:12:37.831 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:37.832 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:37.832 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19163 2026-02-24T18:12:37.833 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T18:12:37.834 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T18:12:37.834 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T18:12:37.834 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-24T18:12:37.835 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_3 td/divergent-priors 2026-02-24T18:12:37.835 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:630: TEST_divergent_3: local dir=td/divergent-priors 2026-02-24T18:12:37.835 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:632: TEST_divergent_3: file_with_random_data 2026-02-24T18:12:37.835 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: file_with_random_data: local size_bytes=512 2026-02-24T18:12:37.835 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: file_with_random_data: mktemp 2026-02-24T18:12:37.836 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: file_with_random_data: local file=/tmp/tmp.paFLOZwKE9 2026-02-24T18:12:37.836 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2374: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.paFLOZwKE9 bs=512 count=1 2026-02-24T18:12:37.837 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records in 2026-02-24T18:12:37.837 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records out 2026-02-24T18:12:37.838 INFO:tasks.workunit.client.0.trial172.stderr:512 bytes copied, 0.000239351 s, 2.1 MB/s 2026-02-24T18:12:37.838 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: file_with_random_data: printf %s /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:37.838 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:632: TEST_divergent_3: local dummyfile=/tmp/tmp.paFLOZwKE9 2026-02-24T18:12:37.838 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:634: TEST_divergent_3: file_with_random_data 1000 2026-02-24T18:12:37.839 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: file_with_random_data: local size_bytes=1000 2026-02-24T18:12:37.839 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: file_with_random_data: mktemp 2026-02-24T18:12:37.840 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: file_with_random_data: local file=/tmp/tmp.9dNuV2JkPI 2026-02-24T18:12:37.840 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2374: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.9dNuV2JkPI bs=1000 count=1 2026-02-24T18:12:37.841 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records in 2026-02-24T18:12:37.841 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records out 2026-02-24T18:12:37.841 INFO:tasks.workunit.client.0.trial172.stderr:1000 bytes (1.0 kB) copied, 0.0001064 s, 9.4 MB/s 2026-02-24T18:12:37.841 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: file_with_random_data: printf %s /tmp/tmp.9dNuV2JkPI 2026-02-24T18:12:37.842 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:634: TEST_divergent_3: local dummyfile2=/tmp/tmp.9dNuV2JkPI 2026-02-24T18:12:37.842 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:636: TEST_divergent_3: local num_osds=3 2026-02-24T18:12:37.842 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:637: TEST_divergent_3: expr 3 - 1 2026-02-24T18:12:37.843 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:637: TEST_divergent_3: seq 0 2 2026-02-24T18:12:37.844 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:637: TEST_divergent_3: local 'osds=0 2026-02-24T18:12:37.844 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:12:37.845 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:12:37.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:638: TEST_divergent_3: run_mon td/divergent-priors a 2026-02-24T18:12:37.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/divergent-priors 2026-02-24T18:12:37.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T18:12:37.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T18:12:37.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T18:12:37.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/divergent-priors/a 2026-02-24T18:12:37.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-24T18:12:37.873 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T18:12:37.873 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:12:37.873 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:12:37.874 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:12:37.874 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:37.874 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:37.874 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:12:37.874 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T18:12:37.911 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T18:12:37.912 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T18:12:37.913 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:12:37.913 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:12:37.913 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T18:12:37.914 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T18:12:37.914 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:12:37.914 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:12:37.914 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:12:37.914 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:12:37.914 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:37.914 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:37.914 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-mon.a.asok 2026-02-24T18:12:37.915 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:12:37.915 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19163/ceph-mon.a.asok config get fsid 2026-02-24T18:12:37.970 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T18:12:37.970 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:12:37.970 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:12:37.970 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T18:12:37.970 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T18:12:37.970 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:12:37.970 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:12:37.970 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:12:37.971 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:12:37.971 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:37.972 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:37.972 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-mon.a.asok 2026-02-24T18:12:37.972 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:12:37.972 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19163/ceph-mon.a.asok config get mon_host 2026-02-24T18:12:38.024 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:639: TEST_divergent_3: run_mgr td/divergent-priors x 2026-02-24T18:12:38.025 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/divergent-priors 2026-02-24T18:12:38.025 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T18:12:38.025 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T18:12:38.025 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T18:12:38.025 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/divergent-priors/x 2026-02-24T18:12:38.025 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T18:12:38.140 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T18:12:38.140 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:12:38.140 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:12:38.140 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:12:38.140 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:38.140 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:38.140 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:12:38.141 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:12:38.142 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:12:38.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: for i in $osds 2026-02-24T18:12:38.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:642: TEST_divergent_3: run_osd td/divergent-priors 0 2026-02-24T18:12:38.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-24T18:12:38.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:12:38.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T18:12:38.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:12:38.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/0 2026-02-24T18:12:38.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:12:38.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:12:38.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:12:38.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:12:38.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-24T18:12:38.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-24T18:12:38.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:12:38.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:12:38.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:12:38.164 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:12:38.164 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:12:38.164 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:12:38.164 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:12:38.164 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:38.164 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:38.164 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:12:38.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:12:38.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:12:38.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:12:38.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:12:38.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:12:38.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:12:38.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:12:38.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:12:38.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:12:38.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:12:38.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:12:38.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:12:38.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/0 2026-02-24T18:12:38.166 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:12:38.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1b2deac1-b721-42fd-ace2-60433262abf1 2026-02-24T18:12:38.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 1b2deac1-b721-42fd-ace2-60433262abf1' 2026-02-24T18:12:38.167 INFO:tasks.workunit.client.0.trial172.stdout:add osd0 1b2deac1-b721-42fd-ace2-60433262abf1 2026-02-24T18:12:38.167 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:12:38.181 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAW6p1pSpRMChAAjF7GUaSMmxy1NIqUGWK57A== 2026-02-24T18:12:38.181 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAW6p1pSpRMChAAjF7GUaSMmxy1NIqUGWK57A=="}' 2026-02-24T18:12:38.181 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1b2deac1-b721-42fd-ace2-60433262abf1 -i td/divergent-priors/0/new.json 2026-02-24T18:12:38.299 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:12:38.307 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/0/new.json 2026-02-24T18:12:38.308 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAW6p1pSpRMChAAjF7GUaSMmxy1NIqUGWK57A== --osd-uuid 1b2deac1-b721-42fd-ace2-60433262abf1 2026-02-24T18:12:38.327 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:38.315+0000 7fcf07ae7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:38.328 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:38.319+0000 7fcf07ae7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:38.329 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:38.319+0000 7fcf07ae7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:38.330 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:38.319+0000 7fcf07ae7a40 -1 bdev(0x55d8362e2400 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-24T18:12:38.330 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:38.319+0000 7fcf07ae7a40 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-24T18:12:39.108 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-24T18:12:39.109 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:12:39.110 INFO:tasks.workunit.client.0.trial172.stdout:adding osd0 key to auth repository 2026-02-24T18:12:39.110 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T18:12:39.110 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:12:39.234 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T18:12:39.235 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:12:39.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:12:39.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:12:39.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:12:39.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:12:39.265 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:39.255+0000 7f16f11ffa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:39.267 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:39.255+0000 7f16f11ffa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:39.269 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:39.259+0000 7f16f11ffa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:39.347 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T18:12:39.347 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:12:39.347 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:12:39.347 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:12:39.347 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:12:39.348 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:12:39.348 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:12:39.348 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:12:39.348 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:12:39.348 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:12:39.469 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:12:39.826 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:39.815+0000 7f16f11ffa40 -1 Falling back to public interface 2026-02-24T18:12:40.470 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:12:40.470 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:12:40.470 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:12:40.471 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:12:40.471 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:12:40.471 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:12:40.681 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:12:40.810 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:40.799+0000 7f16f11ffa40 -1 osd.0 0 log_to_monitors true 2026-02-24T18:12:41.682 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:12:41.682 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:12:41.683 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:12:41.683 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:12:41.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:12:41.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:12:41.894 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3009684553,v1:127.0.0.1:6801/3009684553] [v2:127.0.0.1:6802/3009684553,v1:127.0.0.1:6803/3009684553] exists,up 1b2deac1-b721-42fd-ace2-60433262abf1 2026-02-24T18:12:41.896 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:12:41.896 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: for i in $osds 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:642: TEST_divergent_3: run_osd td/divergent-priors 1 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/1 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:12:41.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:12:41.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:12:41.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:12:41.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:12:41.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:12:41.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/1 2026-02-24T18:12:41.899 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:12:41.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=43112d05-9157-407e-ba40-33f71a9b44d7 2026-02-24T18:12:41.899 INFO:tasks.workunit.client.0.trial172.stdout:add osd1 43112d05-9157-407e-ba40-33f71a9b44d7 2026-02-24T18:12:41.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 43112d05-9157-407e-ba40-33f71a9b44d7' 2026-02-24T18:12:41.900 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:12:41.913 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAZ6p1pq5/9NRAALmzV/nM9L0WzuBNud/4NsA== 2026-02-24T18:12:41.914 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAZ6p1pq5/9NRAALmzV/nM9L0WzuBNud/4NsA=="}' 2026-02-24T18:12:41.914 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 43112d05-9157-407e-ba40-33f71a9b44d7 -i td/divergent-priors/1/new.json 2026-02-24T18:12:42.139 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:12:42.150 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/1/new.json 2026-02-24T18:12:42.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAZ6p1pq5/9NRAALmzV/nM9L0WzuBNud/4NsA== --osd-uuid 43112d05-9157-407e-ba40-33f71a9b44d7 2026-02-24T18:12:42.177 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:42.167+0000 7f6fe3303a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:42.178 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:42.167+0000 7f6fe3303a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:42.179 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:42.167+0000 7f6fe3303a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:42.179 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:42.167+0000 7f6fe3303a40 -1 bdev(0x56051cc84400 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-24T18:12:42.179 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:42.167+0000 7f6fe3303a40 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-24T18:12:42.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-24T18:12:42.887 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:12:42.887 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T18:12:42.888 INFO:tasks.workunit.client.0.trial172.stdout:adding osd1 key to auth repository 2026-02-24T18:12:42.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:12:43.178 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:12:43.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T18:12:43.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:12:43.180 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:12:43.180 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:12:43.180 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:12:43.196 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:43.187+0000 7f320af01a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:43.197 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:43.187+0000 7f320af01a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:43.198 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:43.187+0000 7f320af01a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:43.399 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T18:12:43.400 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:12:43.400 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:12:43.400 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:12:43.400 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:12:43.400 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:12:43.400 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:12:43.400 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:12:43.400 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:12:43.401 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:12:43.615 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:12:43.617 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:43.607+0000 7f320af01a40 -1 Falling back to public interface 2026-02-24T18:12:44.223 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:44.211+0000 7f320af01a40 -1 osd.1 0 log_to_monitors true 2026-02-24T18:12:44.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:12:44.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:12:44.616 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:12:44.617 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:12:44.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:12:44.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:12:44.830 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:12:45.831 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:12:45.831 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:12:45.831 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:12:45.832 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:12:45.832 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:12:45.832 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:12:46.047 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/392529732,v1:127.0.0.1:6811/392529732] [v2:127.0.0.1:6812/392529732,v1:127.0.0.1:6813/392529732] exists,up 43112d05-9157-407e-ba40-33f71a9b44d7 2026-02-24T18:12:46.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:12:46.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:12:46.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:12:46.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: for i in $osds 2026-02-24T18:12:46.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:642: TEST_divergent_3: run_osd td/divergent-priors 2 2026-02-24T18:12:46.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-24T18:12:46.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:12:46.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T18:12:46.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:12:46.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/2 2026-02-24T18:12:46.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:12:46.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:12:46.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:12:46.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:12:46.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-24T18:12:46.052 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-24T18:12:46.052 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:12:46.052 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:12:46.052 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:12:46.052 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:12:46.052 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:12:46.052 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:12:46.052 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:12:46.052 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:12:46.052 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:12:46.052 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:12:46.052 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:12:46.052 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:12:46.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:12:46.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:12:46.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:12:46.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:12:46.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:12:46.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:12:46.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:12:46.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:12:46.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:12:46.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:12:46.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/2 2026-02-24T18:12:46.053 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:12:46.054 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9c16fc6e-c787-4393-91f3-0b8a744ca5a0 2026-02-24T18:12:46.054 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 9c16fc6e-c787-4393-91f3-0b8a744ca5a0' 2026-02-24T18:12:46.055 INFO:tasks.workunit.client.0.trial172.stdout:add osd2 9c16fc6e-c787-4393-91f3-0b8a744ca5a0 2026-02-24T18:12:46.055 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:12:46.066 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAe6p1pxTp/AxAAF9SgPRKHAn7iI6t8G6zcUg== 2026-02-24T18:12:46.066 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAe6p1pxTp/AxAAF9SgPRKHAn7iI6t8G6zcUg=="}' 2026-02-24T18:12:46.066 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9c16fc6e-c787-4393-91f3-0b8a744ca5a0 -i td/divergent-priors/2/new.json 2026-02-24T18:12:46.284 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:12:46.295 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/2/new.json 2026-02-24T18:12:46.297 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAe6p1pxTp/AxAAF9SgPRKHAn7iI6t8G6zcUg== --osd-uuid 9c16fc6e-c787-4393-91f3-0b8a744ca5a0 2026-02-24T18:12:46.323 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:46.311+0000 7f978fbf6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:46.325 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:46.315+0000 7f978fbf6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:46.326 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:46.315+0000 7f978fbf6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:46.326 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:46.315+0000 7f978fbf6a40 -1 bdev(0x5615b2a8e400 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-24T18:12:46.326 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:46.315+0000 7f978fbf6a40 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-24T18:12:46.995 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-24T18:12:46.995 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:12:46.996 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T18:12:46.996 INFO:tasks.workunit.client.0.trial172.stdout:adding osd2 key to auth repository 2026-02-24T18:12:46.997 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:12:47.286 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:12:47.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T18:12:47.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:12:47.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:12:47.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:12:47.288 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:12:47.316 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:47.308+0000 7fddf210ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:47.318 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:47.308+0000 7fddf210ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:47.319 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:47.308+0000 7fddf210ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:12:47.506 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T18:12:47.507 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:12:47.507 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:12:47.507 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:12:47.507 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:12:47.507 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:12:47.507 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:12:47.508 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:12:47.508 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:12:47.508 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:12:47.713 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:47.704+0000 7fddf210ea40 -1 Falling back to public interface 2026-02-24T18:12:47.719 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:12:48.377 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:12:48.368+0000 7fddf210ea40 -1 osd.2 0 log_to_monitors true 2026-02-24T18:12:48.720 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:12:48.720 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:12:48.721 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:12:48.721 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:12:48.721 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:12:48.721 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:12:48.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:12:49.934 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:12:49.935 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:12:49.935 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:12:49.936 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:12:49.936 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:12:49.936 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:12:50.148 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2099556115,v1:127.0.0.1:6819/2099556115] [v2:127.0.0.1:6820/2099556115,v1:127.0.0.1:6821/2099556115] exists,up 9c16fc6e-c787-4393-91f3-0b8a744ca5a0 2026-02-24T18:12:50.149 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:12:50.149 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:12:50.149 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:12:50.149 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:645: TEST_divergent_3: ceph osd set noout 2026-02-24T18:12:50.441 INFO:tasks.workunit.client.0.trial172.stderr:noout is set 2026-02-24T18:12:50.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:646: TEST_divergent_3: ceph osd set noin 2026-02-24T18:12:50.744 INFO:tasks.workunit.client.0.trial172.stderr:noin is set 2026-02-24T18:12:50.755 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:647: TEST_divergent_3: ceph osd set nodown 2026-02-24T18:12:51.050 INFO:tasks.workunit.client.0.trial172.stderr:nodown is set 2026-02-24T18:12:51.064 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:648: TEST_divergent_3: create_pool test 1 1 2026-02-24T18:12:51.064 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-24T18:12:51.305 INFO:tasks.workunit.client.0.trial172.stderr:pool 'test' already exists 2026-02-24T18:12:51.316 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T18:12:52.318 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:649: TEST_divergent_3: ceph osd pool set test size 3 2026-02-24T18:12:52.674 INFO:tasks.workunit.client.0.trial172.stderr:set pool 1 size to 3 2026-02-24T18:12:52.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:650: TEST_divergent_3: ceph osd pool set test min_size 2 2026-02-24T18:12:52.980 INFO:tasks.workunit.client.0.trial172.stderr:set pool 1 min_size to 2 2026-02-24T18:12:52.993 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:653: TEST_divergent_3: ceph osd pool set test pg_autoscale_mode on 2026-02-24T18:12:53.285 INFO:tasks.workunit.client.0.trial172.stderr:set pool 1 pg_autoscale_mode to on 2026-02-24T18:12:53.295 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:655: TEST_divergent_3: divergent=-1 2026-02-24T18:12:53.296 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:656: TEST_divergent_3: date +%s 2026-02-24T18:12:53.297 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:656: TEST_divergent_3: start_time=1771956773 2026-02-24T18:12:53.297 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:657: TEST_divergent_3: max_duration=300 2026-02-24T18:12:53.297 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:659: TEST_divergent_3: '[' -1 -le -1 ']' 2026-02-24T18:12:53.297 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:661: TEST_divergent_3: flush_pg_stats 2026-02-24T18:12:53.297 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:12:53.298 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:12:53.523 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:12:53.523 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:12:53.523 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:12:53.524 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:12:53.524 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:12:53.524 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:12:53.608 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836485 2026-02-24T18:12:53.608 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836485 2026-02-24T18:12:53.609 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836485' 2026-02-24T18:12:53.609 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:12:53.609 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:12:53.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672964 2026-02-24T18:12:53.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672964 2026-02-24T18:12:53.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-24T18:12:53.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:12:53.693 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:12:53.780 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509443 2026-02-24T18:12:53.780 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509443 2026-02-24T18:12:53.780 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-64424509443' 2026-02-24T18:12:53.780 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:12:53.780 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836485 2026-02-24T18:12:53.781 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:12:53.781 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:12:53.782 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836485 2026-02-24T18:12:53.782 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:12:53.783 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836485 2026-02-24T18:12:53.783 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 21474836485 2026-02-24T18:12:53.784 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-24T18:12:53.784 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:12:54.000 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836483 -lt 21474836485 2026-02-24T18:12:54.000 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:12:55.001 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:12:55.002 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:12:55.225 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-24T18:12:55.225 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:12:55.226 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672964 2026-02-24T18:12:55.226 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:12:55.227 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:12:55.228 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672964 2026-02-24T18:12:55.228 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:12:55.229 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672964 2026-02-24T18:12:55.229 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 42949672964 2026-02-24T18:12:55.230 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-24T18:12:55.230 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:12:55.451 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-24T18:12:55.451 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:12:55.452 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509443 2026-02-24T18:12:55.452 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:12:55.453 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:12:55.454 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509443 2026-02-24T18:12:55.454 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:12:55.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509443 2026-02-24T18:12:55.455 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 64424509443 2026-02-24T18:12:55.456 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509443' 2026-02-24T18:12:55.456 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:12:55.679 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509443 -lt 64424509443 2026-02-24T18:12:55.679 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:662: TEST_divergent_3: wait_for_clean 2026-02-24T18:12:55.679 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:12:55.679 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:12:55.679 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:12:55.679 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T18:12:55.680 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:12:55.680 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:12:55.680 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:12:55.681 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:12:55.681 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:12:55.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T18:12:55.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:12:55.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:12:55.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:12:55.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:12:55.777 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:12:55.998 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:12:55.998 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:12:55.998 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:12:55.998 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:12:55.998 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:12:55.998 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:12:56.079 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836487 2026-02-24T18:12:56.079 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836487 2026-02-24T18:12:56.079 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836487' 2026-02-24T18:12:56.079 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:12:56.079 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:12:56.160 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672967 2026-02-24T18:12:56.160 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672967 2026-02-24T18:12:56.160 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836487 1-42949672967' 2026-02-24T18:12:56.161 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:12:56.161 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:12:56.247 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509446 2026-02-24T18:12:56.248 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509446 2026-02-24T18:12:56.248 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836487 1-42949672967 2-64424509446' 2026-02-24T18:12:56.248 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:12:56.248 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836487 2026-02-24T18:12:56.248 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:12:56.249 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:12:56.249 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836487 2026-02-24T18:12:56.250 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:12:56.251 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836487 2026-02-24T18:12:56.251 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 21474836487 2026-02-24T18:12:56.251 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-24T18:12:56.251 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:12:56.467 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-24T18:12:56.467 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:12:57.468 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:12:57.468 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:12:57.691 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-24T18:12:57.692 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:12:57.692 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672967 2026-02-24T18:12:57.692 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:12:57.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:12:57.694 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672967 2026-02-24T18:12:57.694 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:12:57.695 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672967 2026-02-24T18:12:57.696 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 42949672967 2026-02-24T18:12:57.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672967' 2026-02-24T18:12:57.696 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:12:57.910 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672967 -lt 42949672967 2026-02-24T18:12:57.910 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:12:57.910 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509446 2026-02-24T18:12:57.910 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:12:57.912 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:12:57.912 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509446 2026-02-24T18:12:57.912 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:12:57.914 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509446 2026-02-24T18:12:57.914 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 64424509446 2026-02-24T18:12:57.914 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509446' 2026-02-24T18:12:57.914 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:12:58.144 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509446 -lt 64424509446 2026-02-24T18:12:58.145 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:12:58.145 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:12:58.145 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:12:58.447 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:12:58.448 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:12:58.448 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:12:58.448 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:12:58.448 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:12:58.448 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:12:58.448 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:12:58.448 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:12:58.667 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-24T18:12:58.668 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:12:58.668 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:12:58.668 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:12:58.958 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-24T18:12:58.958 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T18:12:58.958 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T18:12:58.958 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:665: TEST_divergent_3: ceph pg dump pgs --format=json 2026-02-24T18:12:58.958 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:665: TEST_divergent_3: jq '.pg_stats[0].up_primary' 2026-02-24T18:12:59.159 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:12:59.168 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:665: TEST_divergent_3: divergent=1 2026-02-24T18:12:59.168 INFO:tasks.workunit.client.0.trial172.stdout:primary and soon to be divergent is 1 2026-02-24T18:12:59.169 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:666: TEST_divergent_3: echo 'primary and soon to be divergent is 1' 2026-02-24T18:12:59.169 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:667: TEST_divergent_3: ceph pg dump pgs 2026-02-24T18:12:59.381 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:12:59.381 INFO:tasks.workunit.client.0.trial172.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T18:12:51.873155+0000 0'0 30:22 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T20:33:27.794667+0000 0 0 2026-02-24T18:12:59.381 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:12:59.381 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:12:59.381 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:12:59.391 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:669: TEST_divergent_3: date +%s 2026-02-24T18:12:59.392 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:669: TEST_divergent_3: current_time=1771956779 2026-02-24T18:12:59.392 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:670: TEST_divergent_3: expr 1771956779 - 1771956773 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:670: TEST_divergent_3: elapsed_time=6 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: '[' 6 -gt 300 ']' 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:659: TEST_divergent_3: '[' 1 -le -1 ']' 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:677: TEST_divergent_3: local non_divergent= 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: for i in $osds 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:680: TEST_divergent_3: '[' 0 = 1 ']' 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:683: TEST_divergent_3: non_divergent=' 0' 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: for i in $osds 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:680: TEST_divergent_3: '[' 1 = 1 ']' 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:681: TEST_divergent_3: continue 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: for i in $osds 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:680: TEST_divergent_3: '[' 2 = 1 ']' 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:683: TEST_divergent_3: non_divergent=' 0 2' 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stdout:writing initial objects 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: echo 'writing initial objects' 2026-02-24T18:12:59.394 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: seq 1 100 2026-02-24T18:12:59.395 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.395 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_1 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.419 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.419 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_2 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.446 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.446 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_3 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_4 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.504 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.504 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_5 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.532 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.532 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_6 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.561 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.561 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_7 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.589 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.589 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_8 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_9 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_10 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.673 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.673 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_11 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.701 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.702 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_12 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.729 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.730 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_13 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.758 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.758 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_14 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.785 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.785 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_15 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.813 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.813 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_16 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.841 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.841 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_17 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.868 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.868 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_18 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_19 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.925 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.925 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_20 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.953 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.953 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_21 /tmp/tmp.paFLOZwKE9 2026-02-24T18:12:59.981 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:12:59.981 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_22 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.001 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.001 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_23 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.030 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.030 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_24 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.052 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_25 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.073 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.073 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_26 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.093 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.094 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_27 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.113 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.114 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_28 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.133 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.134 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_29 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.155 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.155 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_30 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.176 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.176 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_31 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.197 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.197 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_32 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.217 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.217 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_33 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.252 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.252 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_34 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.272 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.272 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_35 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.292 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.292 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_36 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.313 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.313 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_37 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.333 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.333 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_38 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.353 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.353 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_39 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_40 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.394 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_41 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.415 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.415 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_42 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.436 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.436 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_43 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.457 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.457 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_44 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.477 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.477 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_45 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.497 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.498 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_46 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.518 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.518 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_47 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_48 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.560 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.560 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_49 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.581 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.581 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_50 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.605 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.605 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_51 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.625 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.625 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_52 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.646 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.646 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_53 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.666 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.666 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_54 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_55 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.708 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_56 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.734 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.734 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_57 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.757 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.757 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_58 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.782 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.782 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_59 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.810 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.810 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_60 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.838 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.838 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_61 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.866 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.866 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_62 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.894 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.894 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_63 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.926 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.927 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_64 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.954 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.954 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_65 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:00.985 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:00.986 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_66 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.007 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.007 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_67 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.028 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.028 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_68 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.056 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.056 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_69 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.085 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.085 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_70 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.113 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.114 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_71 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.142 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.142 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_72 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.171 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.171 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_73 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.199 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.199 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_74 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.228 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.228 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_75 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_76 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_77 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.313 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.313 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_78 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.341 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.341 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_79 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.369 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.370 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_80 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.397 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.397 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_81 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.425 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.425 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_82 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.453 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.453 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_83 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_84 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.509 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.509 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_85 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_86 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.568 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.568 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_87 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_88 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.625 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.625 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_89 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.653 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.653 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_90 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.681 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.682 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_91 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.711 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.711 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_92 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.740 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.740 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_93 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.768 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.768 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_94 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.796 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.796 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_95 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.825 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.825 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_96 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.853 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.853 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_97 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.881 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_98 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.909 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.909 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_99 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.937 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-24T18:13:01.937 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_100 /tmp/tmp.paFLOZwKE9 2026-02-24T18:13:01.965 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:693: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-24T18:13:01.965 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:693: TEST_divergent_3: wait_for_clean 2026-02-24T18:13:01.965 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:13:01.965 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:13:01.965 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:13:01.966 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-24T18:13:01.966 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:13:01.967 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:13:01.967 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:13:01.967 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:13:01.967 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:13:02.038 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-24T18:13:02.038 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:13:02.038 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:13:02.038 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:13:02.038 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:13:02.038 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:13:02.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:13:02.259 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:13:02.259 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:13:02.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:13:02.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:13:02.259 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:13:02.345 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836491 2026-02-24T18:13:02.345 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836491 2026-02-24T18:13:02.345 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836491' 2026-02-24T18:13:02.345 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:13:02.345 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:13:02.428 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672970 2026-02-24T18:13:02.428 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672970 2026-02-24T18:13:02.429 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836491 1-42949672970' 2026-02-24T18:13:02.429 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:13:02.429 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:13:02.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509449 2026-02-24T18:13:02.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509449 2026-02-24T18:13:02.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449' 2026-02-24T18:13:02.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:13:02.514 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836491 2026-02-24T18:13:02.514 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:13:02.515 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:13:02.516 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836491 2026-02-24T18:13:02.516 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:13:02.517 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836491 2026-02-24T18:13:02.517 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 21474836491 2026-02-24T18:13:02.518 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-24T18:13:02.518 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:13:02.742 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-24T18:13:02.742 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:13:02.743 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672970 2026-02-24T18:13:02.743 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:13:02.744 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:13:02.745 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672970 2026-02-24T18:13:02.745 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:13:02.746 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672970 2026-02-24T18:13:02.746 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-24T18:13:02.746 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 42949672970 2026-02-24T18:13:02.747 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:13:02.978 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672970 -lt 42949672970 2026-02-24T18:13:02.978 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:13:02.979 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509449 2026-02-24T18:13:02.979 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:13:02.980 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:13:02.981 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509449 2026-02-24T18:13:02.981 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:13:02.982 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509449 2026-02-24T18:13:02.982 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 64424509449 2026-02-24T18:13:02.982 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509449' 2026-02-24T18:13:02.983 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:13:03.200 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509447 -lt 64424509449 2026-02-24T18:13:03.200 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:13:04.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:13:04.202 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:13:04.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509447 -lt 64424509449 2026-02-24T18:13:04.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:13:05.425 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T18:13:05.426 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:13:05.642 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509449 -lt 64424509449 2026-02-24T18:13:05.643 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:13:05.643 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:05.644 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:05.923 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:13:05.923 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:13:05.923 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:13:05.923 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:13:05.923 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:13:05.924 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:13:05.924 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:13:05.924 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:13:06.135 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-24T18:13:06.135 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:13:06.135 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:06.135 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:06.419 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-24T18:13:06.419 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T18:13:06.419 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T18:13:06.419 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:695: TEST_divergent_3: get_pg test existing_1 2026-02-24T18:13:06.420 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=test 2026-02-24T18:13:06.420 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=existing_1 2026-02-24T18:13:06.420 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map test existing_1 2026-02-24T18:13:06.420 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-24T18:13:06.635 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:695: TEST_divergent_3: local pgid=1.0 2026-02-24T18:13:06.635 INFO:tasks.workunit.client.0.trial172.stdout:blackholing osds 0 2 2026-02-24T18:13:06.636 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:698: TEST_divergent_3: echo 'blackholing osds 0 2' 2026-02-24T18:13:06.636 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:699: TEST_divergent_3: ceph pg dump pgs 2026-02-24T18:13:06.845 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:13:06.846 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 0 0 0 51200 0 0 16 84 16 active+clean 2026-02-24T18:12:51.873155+0000 30'100 30:122 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T20:33:27.794667+0000 0 0 2026-02-24T18:13:06.846 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:13:06.846 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:13:06.846 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:13:06.855 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: for i in $non_divergent 2026-02-24T18:13:06.856 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:702: TEST_divergent_3: get_asok_path osd.0 2026-02-24T18:13:06.856 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T18:13:06.856 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T18:13:06.857 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:13:06.857 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:13:06.857 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:13:06.857 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.0.asok 2026-02-24T18:13:06.858 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:702: TEST_divergent_3: CEPH_ARGS= 2026-02-24T18:13:06.858 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:702: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-24T18:13:06.911 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:13:06.912 INFO:tasks.workunit.client.0.trial172.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-24T18:13:06.912 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:13:06.917 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: for i in $non_divergent 2026-02-24T18:13:06.919 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:702: TEST_divergent_3: get_asok_path osd.2 2026-02-24T18:13:06.919 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-24T18:13:06.919 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-24T18:13:06.919 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:13:06.919 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:13:06.919 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:13:06.920 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.2.asok 2026-02-24T18:13:06.920 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:702: TEST_divergent_3: CEPH_ARGS= 2026-02-24T18:13:06.920 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:702: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-24T18:13:06.974 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:13:06.974 INFO:tasks.workunit.client.0.trial172.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-24T18:13:06.974 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:13:06.980 INFO:tasks.workunit.client.0.trial172.stdout:create new divergent objects 2026-02-24T18:13:06.981 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:706: TEST_divergent_3: echo 'create new divergent objects' 2026-02-24T18:13:06.981 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:707: TEST_divergent_3: seq 1 2 2026-02-24T18:13:06.981 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:707: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-24T18:13:06.981 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:707: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-24T18:13:06.981 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:709: TEST_divergent_3: rados -p test create newobject_1 2026-02-24T18:13:06.981 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:712: TEST_divergent_3: echo 'writing divergent objects' 2026-02-24T18:13:06.981 INFO:tasks.workunit.client.0.trial172.stdout:writing divergent objects 2026-02-24T18:13:06.981 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:709: TEST_divergent_3: rados -p test create newobject_2 2026-02-24T18:13:06.982 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:713: TEST_divergent_3: seq 1 2 2026-02-24T18:13:06.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:713: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-24T18:13:06.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:713: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-24T18:13:06.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:715: TEST_divergent_3: rados -p test put existing_1 /tmp/tmp.9dNuV2JkPI 2026-02-24T18:13:06.983 INFO:tasks.workunit.client.0.trial172.stdout:remove divergent objects 2026-02-24T18:13:06.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:718: TEST_divergent_3: echo 'remove divergent objects' 2026-02-24T18:13:06.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:715: TEST_divergent_3: rados -p test put existing_2 /tmp/tmp.9dNuV2JkPI 2026-02-24T18:13:06.984 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: seq 1 2 2026-02-24T18:13:06.985 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-24T18:13:06.985 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:721: TEST_divergent_3: expr 1 + 2 2026-02-24T18:13:06.987 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:721: TEST_divergent_3: rmi=3 2026-02-24T18:13:06.987 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-24T18:13:06.987 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:722: TEST_divergent_3: rados -p test rm existing_3 2026-02-24T18:13:06.988 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:721: TEST_divergent_3: expr 2 + 2 2026-02-24T18:13:06.988 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:721: TEST_divergent_3: rmi=4 2026-02-24T18:13:06.989 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:724: TEST_divergent_3: sleep 10 2026-02-24T18:13:06.989 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:722: TEST_divergent_3: rados -p test rm existing_4 2026-02-24T18:13:16.990 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:725: TEST_divergent_3: killall -9 rados 2026-02-24T18:13:17.046 INFO:tasks.workunit.client.0.trial172.stderr:rados: no process found 2026-02-24T18:13:17.046 INFO:tasks.workunit.client.0.trial172.stdout:killing all the osds 2026-02-24T18:13:17.047 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:728: TEST_divergent_3: echo 'killing all the osds' 2026-02-24T18:13:17.047 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:729: TEST_divergent_3: ceph pg dump pgs 2026-02-24T18:13:17.249 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:13:17.250 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 0 0 0 51152 0 0 16 90 16 active+clean 2026-02-24T18:12:51.873155+0000 30'106 30:128 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T20:33:27.794667+0000 0 0 2026-02-24T18:13:17.250 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:13:17.250 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:13:17.250 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:13:17.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:730: TEST_divergent_3: kill_daemons td/divergent-priors KILL osd 2026-02-24T18:13:17.260 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:13:17.260 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:13:17.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:13:17.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:13:17.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:13:17.368 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:13:17.368 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:731: TEST_divergent_3: for i in $osds 2026-02-24T18:13:17.369 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:733: TEST_divergent_3: ceph osd down osd.0 2026-02-24T18:13:17.632 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already down. 2026-02-24T18:13:17.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:731: TEST_divergent_3: for i in $osds 2026-02-24T18:13:17.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:733: TEST_divergent_3: ceph osd down osd.1 2026-02-24T18:13:17.919 INFO:tasks.workunit.client.0.trial172.stderr:osd.1 is already down. 2026-02-24T18:13:17.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:731: TEST_divergent_3: for i in $osds 2026-02-24T18:13:17.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:733: TEST_divergent_3: ceph osd down osd.2 2026-02-24T18:13:18.204 INFO:tasks.workunit.client.0.trial172.stderr:osd.2 is already down. 2026-02-24T18:13:18.214 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:735: TEST_divergent_3: for i in $non_divergent 2026-02-24T18:13:18.214 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:737: TEST_divergent_3: ceph osd out osd.0 2026-02-24T18:13:18.489 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already out. 2026-02-24T18:13:18.498 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:735: TEST_divergent_3: for i in $non_divergent 2026-02-24T18:13:18.498 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:737: TEST_divergent_3: ceph osd out osd.2 2026-02-24T18:13:18.759 INFO:tasks.workunit.client.0.trial172.stderr:osd.2 is already out. 2026-02-24T18:13:18.769 INFO:tasks.workunit.client.0.trial172.stdout:bringing up non_divergent 0 2 2026-02-24T18:13:18.769 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:741: TEST_divergent_3: echo 'bringing up non_divergent 0 2' 2026-02-24T18:13:18.769 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:742: TEST_divergent_3: ceph pg dump pgs 2026-02-24T18:13:18.991 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:13:18.991 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 0 0 0 51152 0 0 16 90 16 stale+active+clean 2026-02-24T18:12:51.873155+0000 30'106 30:128 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T20:33:27.794667+0000 0 0 2026-02-24T18:13:18.992 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:13:18.992 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:13:18.992 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:13:19.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:743: TEST_divergent_3: for i in $non_divergent 2026-02-24T18:13:19.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:745: TEST_divergent_3: activate_osd td/divergent-priors 0 2026-02-24T18:13:19.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:13:19.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:13:19.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-24T18:13:19.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:13:19.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/0 2026-02-24T18:13:19.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:13:19.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:13:19.007 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:13:19.007 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:13:19.007 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-24T18:13:19.007 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-24T18:13:19.007 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:13:19.007 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:13:19.007 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:13:19.007 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:13:19.007 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:13:19.007 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:13:19.007 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:13:19.007 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:13:19.008 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:13:19.008 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:13:19.008 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:13:19.008 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:13:19.008 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:13:19.008 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:13:19.008 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:13:19.008 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:13:19.008 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:13:19.008 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:13:19.008 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:13:19.008 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:13:19.009 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/0 2026-02-24T18:13:19.009 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:13:19.010 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-24T18:13:19.010 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:13:19.010 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/0/whoami 2026-02-24T18:13:19.010 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-24T18:13:19.010 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:13:19.010 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:13:19.010 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:13:19.029 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:13:19.021+0000 7fb698a7ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:13:19.031 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:13:19.025+0000 7fb698a7ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:13:19.032 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:13:19.025+0000 7fb698a7ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:13:19.231 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-24T18:13:19.231 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:13:19.231 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:13:19.232 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:13:19.232 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:13:19.232 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:13:19.232 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:13:19.232 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:13:19.233 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:13:19.233 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:13:19.453 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:13:19.459 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:13:19.453+0000 7fb698a7ba40 -1 Falling back to public interface 2026-02-24T18:13:20.159 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:13:20.153+0000 7fb698a7ba40 -1 osd.0 30 log_to_monitors true 2026-02-24T18:13:20.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:13:20.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:13:20.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:13:20.455 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:13:20.456 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:13:20.456 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:13:20.681 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up out weight 0 up_from 37 up_thru 0 down_at 31 last_clean_interval [5,30) [v2:127.0.0.1:6800/266228372,v1:127.0.0.1:6801/266228372] [v2:127.0.0.1:6802/266228372,v1:127.0.0.1:6803/266228372] exists,up 1b2deac1-b721-42fd-ace2-60433262abf1 2026-02-24T18:13:20.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:13:20.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:13:20.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:13:20.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:743: TEST_divergent_3: for i in $non_divergent 2026-02-24T18:13:20.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:745: TEST_divergent_3: activate_osd td/divergent-priors 2 2026-02-24T18:13:20.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:13:20.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:13:20.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T18:13:20.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:13:20.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/2 2026-02-24T18:13:20.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:13:20.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:13:20.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:13:20.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:13:20.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-24T18:13:20.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-24T18:13:20.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:13:20.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:13:20.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:13:20.686 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:13:20.686 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:13:20.686 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:13:20.686 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:13:20.687 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:13:20.687 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:13:20.687 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:13:20.687 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:13:20.687 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:13:20.687 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:13:20.687 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:13:20.687 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:13:20.687 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:13:20.687 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:13:20.687 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:13:20.687 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:13:20.688 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:13:20.688 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/2 2026-02-24T18:13:20.688 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:13:20.689 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T18:13:20.689 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:13:20.689 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/2/whoami 2026-02-24T18:13:20.689 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T18:13:20.689 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:13:20.690 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:13:20.690 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:13:20.703 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:13:20.697+0000 7f22e3f68a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:13:20.705 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:13:20.697+0000 7f22e3f68a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:13:20.706 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:13:20.701+0000 7f22e3f68a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:13:20.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T18:13:20.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:13:20.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:13:20.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:13:20.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:13:20.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:13:20.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:13:20.899 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:13:20.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:13:20.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:13:21.127 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:13:21.127 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:13:21.121+0000 7f22e3f68a40 -1 Falling back to public interface 2026-02-24T18:13:21.751 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:13:21.745+0000 7f22e3f68a40 -1 osd.2 30 log_to_monitors true 2026-02-24T18:13:22.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:13:22.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:13:22.128 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:13:22.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:13:22.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:13:22.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:13:22.343 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up out weight 0 up_from 39 up_thru 0 down_at 33 last_clean_interval [15,30) [v2:127.0.0.1:6810/220259212,v1:127.0.0.1:6811/220259212] [v2:127.0.0.1:6812/220259212,v1:127.0.0.1:6813/220259212] exists,up 9c16fc6e-c787-4393-91f3-0b8a744ca5a0 2026-02-24T18:13:22.343 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:13:22.343 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:13:22.343 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:13:22.343 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:747: TEST_divergent_3: for i in $non_divergent 2026-02-24T18:13:22.343 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:749: TEST_divergent_3: ceph osd in osd.0 2026-02-24T18:13:22.594 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already in. 2026-02-24T18:13:22.605 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:747: TEST_divergent_3: for i in $non_divergent 2026-02-24T18:13:22.606 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:749: TEST_divergent_3: ceph osd in osd.2 2026-02-24T18:13:22.872 INFO:tasks.workunit.client.0.trial172.stderr:osd.2 is already in. 2026-02-24T18:13:22.882 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:752: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-24T18:13:22.883 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:752: TEST_divergent_3: wait_for_clean 2026-02-24T18:13:22.883 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:13:22.883 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:13:22.883 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:13:22.883 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-24T18:13:22.884 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:13:22.884 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:13:22.885 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:13:22.885 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:13:22.885 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:13:22.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-24T18:13:22.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:13:22.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:13:22.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:13:22.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:13:22.945 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:13:23.173 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:13:23.173 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:13:23.173 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:13:23.174 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:13:23.174 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:13:23.174 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:13:23.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=158913789955 2026-02-24T18:13:23.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 158913789955 2026-02-24T18:13:23.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-158913789955' 2026-02-24T18:13:23.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:13:23.257 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:13:23.321 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq= 2026-02-24T18:13:23.321 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: true 2026-02-24T18:13:23.322 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z '' 2026-02-24T18:13:23.322 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: flush_pg_stats: continue 2026-02-24T18:13:23.322 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:13:23.322 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:13:23.399 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=167503724547 2026-02-24T18:13:23.400 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 167503724547 2026-02-24T18:13:23.400 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-158913789955 2-167503724547' 2026-02-24T18:13:23.400 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:13:23.400 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-158913789955 2026-02-24T18:13:23.400 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:13:23.401 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:13:23.402 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-158913789955 2026-02-24T18:13:23.402 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:13:23.404 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=158913789955 2026-02-24T18:13:23.404 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 158913789955' 2026-02-24T18:13:23.404 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 158913789955 2026-02-24T18:13:23.404 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:13:23.642 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 1 -lt 158913789955 2026-02-24T18:13:23.642 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:13:24.643 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:13:24.644 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:13:24.865 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789955 -lt 158913789955 2026-02-24T18:13:24.865 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:13:24.866 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-167503724547 2026-02-24T18:13:24.866 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:13:24.867 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:13:24.868 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-167503724547 2026-02-24T18:13:24.868 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:13:24.869 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=167503724547 2026-02-24T18:13:24.870 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 167503724547 2026-02-24T18:13:24.870 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 167503724547' 2026-02-24T18:13:24.870 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:13:25.092 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-24T18:13:25.093 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:13:25.093 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:25.093 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:25.387 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:13:25.387 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:13:25.387 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:13:25.387 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:13:25.387 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:13:25.387 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:13:25.388 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:13:25.388 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:13:25.609 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:13:25.610 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:13:25.610 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:25.610 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:25.904 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:13:25.904 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-24T18:13:25.904 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-24T18:13:25.904 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-24T18:13:25.904 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:13:25.904 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-24T18:13:26.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:13:26.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:13:26.006 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:13:26.006 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:13:26.006 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:13:26.006 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:13:26.007 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:13:26.007 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:13:26.227 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:13:26.228 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:13:26.228 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:26.228 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:26.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:13:26.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:13:26.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:13:26.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:13:26.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:13:26.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:13:26.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:13:26.514 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:13:26.514 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:13:26.805 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:13:26.805 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:13:26.805 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 8 )) 2026-02-24T18:13:26.806 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:13:26.806 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-24T18:13:27.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:13:27.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:13:27.008 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:13:27.008 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:13:27.008 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:13:27.008 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:13:27.008 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:13:27.008 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:13:27.216 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:13:27.216 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:13:27.217 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:27.217 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:27.500 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:13:27.501 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:13:27.501 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:13:27.501 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:13:27.501 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:13:27.501 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:13:27.501 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:13:27.501 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:13:27.501 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:13:27.775 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:13:27.775 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:13:27.775 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 8 )) 2026-02-24T18:13:27.775 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:13:27.776 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-24T18:13:28.176 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:13:28.176 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:13:28.177 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:13:28.177 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:13:28.177 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:13:28.177 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:13:28.177 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:13:28.177 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:13:28.391 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:13:28.392 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:13:28.392 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:28.392 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:28.670 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:13:28.670 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:13:28.670 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:13:28.670 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:13:28.670 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:13:28.670 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:13:28.671 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:13:28.671 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:13:28.671 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:13:28.953 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:13:28.954 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:13:28.954 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 8 )) 2026-02-24T18:13:28.954 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:13:28.954 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-24T18:13:29.754 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:13:29.754 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:13:29.755 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:13:29.755 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:13:29.755 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:13:29.755 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:13:29.755 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:13:29.755 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:13:29.968 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:13:29.968 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:13:29.969 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:29.969 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:30.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:13:30.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:13:30.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:13:30.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:13:30.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:13:30.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:13:30.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:13:30.262 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:13:30.262 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:13:30.559 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:13:30.559 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:13:30.559 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 8 )) 2026-02-24T18:13:30.559 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:13:30.560 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 1.6 2026-02-24T18:13:32.161 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:13:32.161 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:13:32.162 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:13:32.162 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:13:32.162 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:13:32.162 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:13:32.162 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:13:32.162 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:13:32.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:13:32.377 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:13:32.377 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:32.377 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:32.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:13:32.659 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:13:32.659 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:13:32.659 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:13:32.659 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:13:32.659 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:13:32.659 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:13:32.659 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:13:32.659 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:13:32.937 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:13:32.937 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:13:32.937 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 5 >= 8 )) 2026-02-24T18:13:32.937 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:13:32.937 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 3.2 2026-02-24T18:13:36.137 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:13:36.137 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:13:36.138 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:13:36.138 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:13:36.138 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:13:36.138 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:13:36.138 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:13:36.138 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:13:36.352 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:13:36.352 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:13:36.353 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:36.353 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:36.635 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:13:36.635 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:13:36.635 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:13:36.635 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:13:36.635 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:13:36.635 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:13:36.635 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:13:36.635 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:13:36.635 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:13:36.919 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:13:36.919 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:13:36.919 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 6 >= 8 )) 2026-02-24T18:13:36.919 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:13:36.919 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 6.4 2026-02-24T18:13:43.320 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:13:43.320 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:13:43.321 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:13:43.321 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:13:43.321 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:13:43.322 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:13:43.322 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:13:43.322 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:13:43.533 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:13:43.534 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:13:43.534 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:43.534 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:43.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:13:43.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:13:43.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:13:43.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:13:43.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:13:43.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:13:43.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:13:43.820 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:13:43.820 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:13:44.094 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:13:44.095 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:13:44.095 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 7 >= 8 )) 2026-02-24T18:13:44.095 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:13:44.095 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 7.3 2026-02-24T18:13:51.395 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:13:51.395 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:13:51.396 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:13:51.396 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:13:51.397 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:13:51.397 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:13:51.397 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:13:51.397 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:13:51.607 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:13:51.607 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:13:51.607 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:51.608 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:51.887 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-24T18:13:51.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:13:51.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:13:51.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:13:51.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:13:51.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:13:51.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:13:51.888 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:13:51.888 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:13:52.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:13:52.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:13:52.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 8 >= 8 )) 2026-02-24T18:13:52.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-24T18:13:52.437 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:13:52.438 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_fingerprint": "09dc039b-8fcf-4bae-aa5e-951070c69b4f", 2026-02-24T18:13:52.438 INFO:tasks.workunit.client.0.trial172.stdout: "version": "20.3.0-5507-g618c318a", 2026-02-24T18:13:52.438 INFO:tasks.workunit.client.0.trial172.stdout: "commit": "618c318a7a0b287ab5aeff207b7bf1d73ef45044", 2026-02-24T18:13:52.438 INFO:tasks.workunit.client.0.trial172.stdout: "timestamp": "2026-02-24T18:13:52.433558+0000", 2026-02-24T18:13:52.438 INFO:tasks.workunit.client.0.trial172.stdout: "tag": "", 2026-02-24T18:13:52.439 INFO:tasks.workunit.client.0.trial172.stdout: "health": { 2026-02-24T18:13:52.439 INFO:tasks.workunit.client.0.trial172.stdout: "status": "HEALTH_WARN", 2026-02-24T18:13:52.439 INFO:tasks.workunit.client.0.trial172.stdout: "checks": { 2026-02-24T18:13:52.439 INFO:tasks.workunit.client.0.trial172.stdout: "OSDMAP_FLAGS": { 2026-02-24T18:13:52.439 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:13:52.439 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:13:52.439 INFO:tasks.workunit.client.0.trial172.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-24T18:13:52.439 INFO:tasks.workunit.client.0.trial172.stdout: "count": 17 2026-02-24T18:13:52.440 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.440 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [], 2026-02-24T18:13:52.440 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:13:52.440 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.440 INFO:tasks.workunit.client.0.trial172.stdout: "OSD_DOWN": { 2026-02-24T18:13:52.440 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:13:52.440 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:13:52.440 INFO:tasks.workunit.client.0.trial172.stdout: "message": "1 osds down", 2026-02-24T18:13:52.441 INFO:tasks.workunit.client.0.trial172.stdout: "count": 1 2026-02-24T18:13:52.441 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.441 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [ 2026-02-24T18:13:52.441 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.441 INFO:tasks.workunit.client.0.trial172.stdout: "message": "osd.1 (root=default,host=trial172) is down" 2026-02-24T18:13:52.441 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.441 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.441 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:13:52.442 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.442 INFO:tasks.workunit.client.0.trial172.stdout: "PG_DEGRADED": { 2026-02-24T18:13:52.442 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:13:52.442 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:13:52.442 INFO:tasks.workunit.client.0.trial172.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 32 pgs degraded", 2026-02-24T18:13:52.442 INFO:tasks.workunit.client.0.trial172.stdout: "count": 32 2026-02-24T18:13:52.442 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.442 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [ 2026-02-24T18:13:52.443 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.443 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.0 is active+undersized+degraded+wait, acting [0,2]" 2026-02-24T18:13:52.443 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.443 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.443 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.1 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.443 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.443 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.443 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.2 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.444 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.444 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.444 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.3 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.444 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.444 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.444 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.4 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.444 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.444 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.444 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.5 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.444 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.444 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.444 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.6 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.444 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.445 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.445 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.7 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.445 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.445 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.445 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.8 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.445 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.445 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.445 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.9 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.445 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.445 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.445 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.a is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.445 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.445 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.445 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.b is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.446 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.446 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.446 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.c is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.446 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.446 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.446 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.d is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.446 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.446 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.446 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.e is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.446 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.446 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.446 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.f is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.447 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.447 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.447 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.10 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.447 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.447 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.447 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.11 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.447 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.447 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.447 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.12 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.447 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.447 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.447 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.13 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.447 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.447 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.448 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.14 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.448 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.448 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.448 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.15 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.448 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.448 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.448 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.16 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.448 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.448 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.448 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.17 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.448 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.448 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.448 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.18 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.449 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.449 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.449 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.19 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.449 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.449 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.449 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.1a is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.449 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.449 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.449 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.1b is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.449 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.449 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.449 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.1c is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.449 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.450 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.450 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.1d is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.450 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.450 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.450 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.1e is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.450 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.450 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.450 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.1f is active+undersized+degraded, acting [0,2]" 2026-02-24T18:13:52.450 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.450 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.450 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:13:52.450 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.450 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.451 INFO:tasks.workunit.client.0.trial172.stdout: "mutes": [] 2026-02-24T18:13:52.451 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.451 INFO:tasks.workunit.client.0.trial172.stdout: "monmap_first_committed": 1, 2026-02-24T18:13:52.451 INFO:tasks.workunit.client.0.trial172.stdout: "monmap_last_committed": 1, 2026-02-24T18:13:52.451 INFO:tasks.workunit.client.0.trial172.stdout: "monmap": { 2026-02-24T18:13:52.451 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 1, 2026-02-24T18:13:52.451 INFO:tasks.workunit.client.0.trial172.stdout: "fsid": "27512777-e37f-4f16-b176-1621842bd4df", 2026-02-24T18:13:52.451 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:12:37.858142Z", 2026-02-24T18:13:52.451 INFO:tasks.workunit.client.0.trial172.stdout: "created": "2026-02-24T18:12:37.858142Z", 2026-02-24T18:13:52.451 INFO:tasks.workunit.client.0.trial172.stdout: "min_mon_release": 20, 2026-02-24T18:13:52.451 INFO:tasks.workunit.client.0.trial172.stdout: "min_mon_release_name": "tentacle", 2026-02-24T18:13:52.451 INFO:tasks.workunit.client.0.trial172.stdout: "election_strategy": 1, 2026-02-24T18:13:52.451 INFO:tasks.workunit.client.0.trial172.stdout: "disallowed_leaders": "", 2026-02-24T18:13:52.452 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode": false, 2026-02-24T18:13:52.452 INFO:tasks.workunit.client.0.trial172.stdout: "tiebreaker_mon": "", 2026-02-24T18:13:52.452 INFO:tasks.workunit.client.0.trial172.stdout: "removed_ranks": "", 2026-02-24T18:13:52.452 INFO:tasks.workunit.client.0.trial172.stdout: "features": { 2026-02-24T18:13:52.452 INFO:tasks.workunit.client.0.trial172.stdout: "persistent": [ 2026-02-24T18:13:52.452 INFO:tasks.workunit.client.0.trial172.stdout: "kraken", 2026-02-24T18:13:52.452 INFO:tasks.workunit.client.0.trial172.stdout: "luminous", 2026-02-24T18:13:52.452 INFO:tasks.workunit.client.0.trial172.stdout: "mimic", 2026-02-24T18:13:52.452 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap-prune", 2026-02-24T18:13:52.452 INFO:tasks.workunit.client.0.trial172.stdout: "nautilus", 2026-02-24T18:13:52.452 INFO:tasks.workunit.client.0.trial172.stdout: "octopus", 2026-02-24T18:13:52.452 INFO:tasks.workunit.client.0.trial172.stdout: "pacific", 2026-02-24T18:13:52.452 INFO:tasks.workunit.client.0.trial172.stdout: "elector-pinging", 2026-02-24T18:13:52.453 INFO:tasks.workunit.client.0.trial172.stdout: "quincy", 2026-02-24T18:13:52.453 INFO:tasks.workunit.client.0.trial172.stdout: "reef", 2026-02-24T18:13:52.453 INFO:tasks.workunit.client.0.trial172.stdout: "squid", 2026-02-24T18:13:52.453 INFO:tasks.workunit.client.0.trial172.stdout: "tentacle", 2026-02-24T18:13:52.453 INFO:tasks.workunit.client.0.trial172.stdout: "nvmeof_beacon_diff" 2026-02-24T18:13:52.453 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.453 INFO:tasks.workunit.client.0.trial172.stdout: "optional": [] 2026-02-24T18:13:52.453 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.453 INFO:tasks.workunit.client.0.trial172.stdout: "mons": [ 2026-02-24T18:13:52.453 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.453 INFO:tasks.workunit.client.0.trial172.stdout: "rank": 0, 2026-02-24T18:13:52.453 INFO:tasks.workunit.client.0.trial172.stdout: "name": "a", 2026-02-24T18:13:52.453 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:13:52.453 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:13:52.454 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.454 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:13:52.454 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:7115", 2026-02-24T18:13:52.454 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 0 2026-02-24T18:13:52.454 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.454 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.454 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.454 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:7115/0", 2026-02-24T18:13:52.454 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-24T18:13:52.454 INFO:tasks.workunit.client.0.trial172.stdout: "priority": 0, 2026-02-24T18:13:52.454 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 0, 2026-02-24T18:13:52.454 INFO:tasks.workunit.client.0.trial172.stdout: "time_added": "2026-02-24T18:12:37:858137+0000", 2026-02-24T18:13:52.455 INFO:tasks.workunit.client.0.trial172.stdout: "crush_location": [] 2026-02-24T18:13:52.455 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.455 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.455 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.455 INFO:tasks.workunit.client.0.trial172.stdout: "quorum": [ 2026-02-24T18:13:52.455 INFO:tasks.workunit.client.0.trial172.stdout: 0 2026-02-24T18:13:52.455 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.455 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap": { 2026-02-24T18:13:52.455 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 45, 2026-02-24T18:13:52.455 INFO:tasks.workunit.client.0.trial172.stdout: "fsid": "27512777-e37f-4f16-b176-1621842bd4df", 2026-02-24T18:13:52.455 INFO:tasks.workunit.client.0.trial172.stdout: "created": "2026-02-24T18:12:37.946691+0000", 2026-02-24T18:13:52.455 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:13:42.573809+0000", 2026-02-24T18:13:52.455 INFO:tasks.workunit.client.0.trial172.stdout: "last_up_change": "2026-02-24T18:13:21.902285+0000", 2026-02-24T18:13:52.456 INFO:tasks.workunit.client.0.trial172.stdout: "last_in_change": "2026-02-24T18:13:22.809612+0000", 2026-02-24T18:13:52.456 INFO:tasks.workunit.client.0.trial172.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-24T18:13:52.456 INFO:tasks.workunit.client.0.trial172.stdout: "flags_num": 5800384, 2026-02-24T18:13:52.456 INFO:tasks.workunit.client.0.trial172.stdout: "flags_set": [ 2026-02-24T18:13:52.456 INFO:tasks.workunit.client.0.trial172.stdout: "nodown", 2026-02-24T18:13:52.456 INFO:tasks.workunit.client.0.trial172.stdout: "noin", 2026-02-24T18:13:52.456 INFO:tasks.workunit.client.0.trial172.stdout: "noout", 2026-02-24T18:13:52.456 INFO:tasks.workunit.client.0.trial172.stdout: "pglog_hardlimit", 2026-02-24T18:13:52.456 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snapdirs", 2026-02-24T18:13:52.456 INFO:tasks.workunit.client.0.trial172.stdout: "recovery_deletes", 2026-02-24T18:13:52.456 INFO:tasks.workunit.client.0.trial172.stdout: "sortbitwise" 2026-02-24T18:13:52.456 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.456 INFO:tasks.workunit.client.0.trial172.stdout: "crush_version": 9, 2026-02-24T18:13:52.456 INFO:tasks.workunit.client.0.trial172.stdout: "full_ratio": 0.99000000953674316, 2026-02-24T18:13:52.457 INFO:tasks.workunit.client.0.trial172.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-24T18:13:52.457 INFO:tasks.workunit.client.0.trial172.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-24T18:13:52.457 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_snapshot": "", 2026-02-24T18:13:52.457 INFO:tasks.workunit.client.0.trial172.stdout: "pool_max": 1, 2026-02-24T18:13:52.457 INFO:tasks.workunit.client.0.trial172.stdout: "max_osd": 3, 2026-02-24T18:13:52.457 INFO:tasks.workunit.client.0.trial172.stdout: "require_min_compat_client": "luminous", 2026-02-24T18:13:52.457 INFO:tasks.workunit.client.0.trial172.stdout: "min_compat_client": "jewel", 2026-02-24T18:13:52.457 INFO:tasks.workunit.client.0.trial172.stdout: "require_osd_release": "tentacle", 2026-02-24T18:13:52.457 INFO:tasks.workunit.client.0.trial172.stdout: "allow_crimson": false, 2026-02-24T18:13:52.457 INFO:tasks.workunit.client.0.trial172.stdout: "pools": [ 2026-02-24T18:13:52.457 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.457 INFO:tasks.workunit.client.0.trial172.stdout: "pool": 1, 2026-02-24T18:13:52.458 INFO:tasks.workunit.client.0.trial172.stdout: "pool_name": "test", 2026-02-24T18:13:52.458 INFO:tasks.workunit.client.0.trial172.stdout: "create_time": "2026-02-24T18:12:51.223307+0000", 2026-02-24T18:13:52.458 INFO:tasks.workunit.client.0.trial172.stdout: "flags": 1, 2026-02-24T18:13:52.458 INFO:tasks.workunit.client.0.trial172.stdout: "flags_names": "hashpspool", 2026-02-24T18:13:52.458 INFO:tasks.workunit.client.0.trial172.stdout: "type": 1, 2026-02-24T18:13:52.458 INFO:tasks.workunit.client.0.trial172.stdout: "size": 3, 2026-02-24T18:13:52.458 INFO:tasks.workunit.client.0.trial172.stdout: "min_size": 2, 2026-02-24T18:13:52.458 INFO:tasks.workunit.client.0.trial172.stdout: "crush_rule": 0, 2026-02-24T18:13:52.458 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_count": 0, 2026-02-24T18:13:52.458 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_target": 0, 2026-02-24T18:13:52.458 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_barrier": 0, 2026-02-24T18:13:52.458 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-24T18:13:52.458 INFO:tasks.workunit.client.0.trial172.stdout: "is_stretch_pool": false, 2026-02-24T18:13:52.459 INFO:tasks.workunit.client.0.trial172.stdout: "object_hash": 2, 2026-02-24T18:13:52.459 INFO:tasks.workunit.client.0.trial172.stdout: "pg_autoscale_mode": "on", 2026-02-24T18:13:52.459 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num": 32, 2026-02-24T18:13:52.459 INFO:tasks.workunit.client.0.trial172.stdout: "pg_placement_num": 1, 2026-02-24T18:13:52.459 INFO:tasks.workunit.client.0.trial172.stdout: "pg_placement_num_target": 32, 2026-02-24T18:13:52.459 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num_target": 32, 2026-02-24T18:13:52.459 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num_pending": 32, 2026-02-24T18:13:52.459 INFO:tasks.workunit.client.0.trial172.stdout: "last_pg_merge_meta": { 2026-02-24T18:13:52.459 INFO:tasks.workunit.client.0.trial172.stdout: "source_pgid": "0.0", 2026-02-24T18:13:52.459 INFO:tasks.workunit.client.0.trial172.stdout: "ready_epoch": 0, 2026-02-24T18:13:52.459 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_started": 0, 2026-02-24T18:13:52.459 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_clean": 0, 2026-02-24T18:13:52.459 INFO:tasks.workunit.client.0.trial172.stdout: "source_version": "0'0", 2026-02-24T18:13:52.460 INFO:tasks.workunit.client.0.trial172.stdout: "target_version": "0'0" 2026-02-24T18:13:52.460 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.460 INFO:tasks.workunit.client.0.trial172.stdout: "last_change": "44", 2026-02-24T18:13:52.460 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend": "0", 2026-02-24T18:13:52.460 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-24T18:13:52.460 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend_preluminous": "44", 2026-02-24T18:13:52.460 INFO:tasks.workunit.client.0.trial172.stdout: "auid": 0, 2026-02-24T18:13:52.460 INFO:tasks.workunit.client.0.trial172.stdout: "snap_mode": "selfmanaged", 2026-02-24T18:13:52.460 INFO:tasks.workunit.client.0.trial172.stdout: "snap_seq": 0, 2026-02-24T18:13:52.460 INFO:tasks.workunit.client.0.trial172.stdout: "snap_epoch": 0, 2026-02-24T18:13:52.460 INFO:tasks.workunit.client.0.trial172.stdout: "pool_snaps": [], 2026-02-24T18:13:52.460 INFO:tasks.workunit.client.0.trial172.stdout: "removed_snaps": "[]", 2026-02-24T18:13:52.460 INFO:tasks.workunit.client.0.trial172.stdout: "quota_max_bytes": 0, 2026-02-24T18:13:52.461 INFO:tasks.workunit.client.0.trial172.stdout: "quota_max_objects": 0, 2026-02-24T18:13:52.461 INFO:tasks.workunit.client.0.trial172.stdout: "tiers": [], 2026-02-24T18:13:52.461 INFO:tasks.workunit.client.0.trial172.stdout: "tier_of": -1, 2026-02-24T18:13:52.461 INFO:tasks.workunit.client.0.trial172.stdout: "read_tier": -1, 2026-02-24T18:13:52.461 INFO:tasks.workunit.client.0.trial172.stdout: "write_tier": -1, 2026-02-24T18:13:52.461 INFO:tasks.workunit.client.0.trial172.stdout: "cache_mode": "none", 2026-02-24T18:13:52.461 INFO:tasks.workunit.client.0.trial172.stdout: "target_max_bytes": 0, 2026-02-24T18:13:52.461 INFO:tasks.workunit.client.0.trial172.stdout: "target_max_objects": 0, 2026-02-24T18:13:52.461 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-24T18:13:52.461 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-24T18:13:52.461 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-24T18:13:52.461 INFO:tasks.workunit.client.0.trial172.stdout: "cache_min_flush_age": 0, 2026-02-24T18:13:52.461 INFO:tasks.workunit.client.0.trial172.stdout: "cache_min_evict_age": 0, 2026-02-24T18:13:52.461 INFO:tasks.workunit.client.0.trial172.stdout: "erasure_code_profile": "", 2026-02-24T18:13:52.462 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_params": { 2026-02-24T18:13:52.462 INFO:tasks.workunit.client.0.trial172.stdout: "type": "none" 2026-02-24T18:13:52.462 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.462 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_period": 0, 2026-02-24T18:13:52.462 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_count": 0, 2026-02-24T18:13:52.462 INFO:tasks.workunit.client.0.trial172.stdout: "use_gmt_hitset": true, 2026-02-24T18:13:52.462 INFO:tasks.workunit.client.0.trial172.stdout: "min_read_recency_for_promote": 0, 2026-02-24T18:13:52.462 INFO:tasks.workunit.client.0.trial172.stdout: "min_write_recency_for_promote": 0, 2026-02-24T18:13:52.462 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_grade_decay_rate": 0, 2026-02-24T18:13:52.462 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_search_last_n": 0, 2026-02-24T18:13:52.462 INFO:tasks.workunit.client.0.trial172.stdout: "grade_table": [], 2026-02-24T18:13:52.462 INFO:tasks.workunit.client.0.trial172.stdout: "stripe_width": 0, 2026-02-24T18:13:52.462 INFO:tasks.workunit.client.0.trial172.stdout: "expected_num_objects": 0, 2026-02-24T18:13:52.463 INFO:tasks.workunit.client.0.trial172.stdout: "fast_read": false, 2026-02-24T18:13:52.463 INFO:tasks.workunit.client.0.trial172.stdout: "nonprimary_shards": "{}", 2026-02-24T18:13:52.463 INFO:tasks.workunit.client.0.trial172.stdout: "options": {}, 2026-02-24T18:13:52.463 INFO:tasks.workunit.client.0.trial172.stdout: "application_metadata": {}, 2026-02-24T18:13:52.463 INFO:tasks.workunit.client.0.trial172.stdout: "read_balance": { 2026-02-24T18:13:52.463 INFO:tasks.workunit.client.0.trial172.stdout: "score_type": "Fair distribution", 2026-02-24T18:13:52.463 INFO:tasks.workunit.client.0.trial172.stdout: "score_acting": 2.9900000095367432, 2026-02-24T18:13:52.463 INFO:tasks.workunit.client.0.trial172.stdout: "score_stable": 2.9900000095367432, 2026-02-24T18:13:52.463 INFO:tasks.workunit.client.0.trial172.stdout: "optimal_score": 0.67000001668930054, 2026-02-24T18:13:52.463 INFO:tasks.workunit.client.0.trial172.stdout: "raw_score_acting": 2, 2026-02-24T18:13:52.463 INFO:tasks.workunit.client.0.trial172.stdout: "raw_score_stable": 2, 2026-02-24T18:13:52.463 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity_weighted": 1, 2026-02-24T18:13:52.463 INFO:tasks.workunit.client.0.trial172.stdout: "average_primary_affinity": 1, 2026-02-24T18:13:52.464 INFO:tasks.workunit.client.0.trial172.stdout: "average_primary_affinity_weighted": 1 2026-02-24T18:13:52.464 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.464 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.464 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.464 INFO:tasks.workunit.client.0.trial172.stdout: "osds": [ 2026-02-24T18:13:52.464 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.464 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:13:52.464 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "1b2deac1-b721-42fd-ace2-60433262abf1", 2026-02-24T18:13:52.464 INFO:tasks.workunit.client.0.trial172.stdout: "up": 1, 2026-02-24T18:13:52.464 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:13:52.464 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:13:52.464 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:13:52.464 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 5, 2026-02-24T18:13:52.465 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 30, 2026-02-24T18:13:52.465 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 37, 2026-02-24T18:13:52.465 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 44, 2026-02-24T18:13:52.465 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 31, 2026-02-24T18:13:52.465 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:13:52.465 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:13:52.465 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:13:52.465 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.465 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:13:52.465 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6800", 2026-02-24T18:13:52.465 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:13:52.465 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.465 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.466 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:13:52.466 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6801", 2026-02-24T18:13:52.466 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:13:52.466 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.466 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.466 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.466 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:13:52.466 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:13:52.466 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.466 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:13:52.466 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6802", 2026-02-24T18:13:52.466 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:13:52.466 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.467 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.467 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:13:52.467 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6803", 2026-02-24T18:13:52.467 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:13:52.467 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.467 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.467 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.467 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:13:52.467 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:13:52.467 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.467 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:13:52.467 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6806", 2026-02-24T18:13:52.467 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:13:52.468 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.468 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.468 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:13:52.468 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6807", 2026-02-24T18:13:52.468 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:13:52.468 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.468 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.468 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.468 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:13:52.468 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:13:52.468 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.468 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:13:52.468 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6804", 2026-02-24T18:13:52.468 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:13:52.469 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.469 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.469 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:13:52.469 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6805", 2026-02-24T18:13:52.469 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:13:52.469 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.469 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.469 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.469 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6801/266228372", 2026-02-24T18:13:52.469 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6803/266228372", 2026-02-24T18:13:52.469 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6807/266228372", 2026-02-24T18:13:52.469 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6805/266228372", 2026-02-24T18:13:52.469 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:13:52.470 INFO:tasks.workunit.client.0.trial172.stdout: "exists", 2026-02-24T18:13:52.470 INFO:tasks.workunit.client.0.trial172.stdout: "up" 2026-02-24T18:13:52.470 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.470 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.470 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.470 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:13:52.470 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "43112d05-9157-407e-ba40-33f71a9b44d7", 2026-02-24T18:13:52.470 INFO:tasks.workunit.client.0.trial172.stdout: "up": 0, 2026-02-24T18:13:52.470 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:13:52.470 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:13:52.470 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:13:52.470 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 0, 2026-02-24T18:13:52.470 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 0, 2026-02-24T18:13:52.471 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 10, 2026-02-24T18:13:52.471 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 22, 2026-02-24T18:13:52.471 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 32, 2026-02-24T18:13:52.471 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:13:52.471 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:13:52.471 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:13:52.471 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.471 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:13:52.471 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6810", 2026-02-24T18:13:52.471 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:13:52.471 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.471 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.471 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:13:52.471 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6811", 2026-02-24T18:13:52.471 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:13:52.472 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.472 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.472 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.472 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:13:52.472 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:13:52.472 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.472 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:13:52.472 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6812", 2026-02-24T18:13:52.472 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:13:52.472 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.472 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.472 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:13:52.472 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6813", 2026-02-24T18:13:52.472 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:13:52.472 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.473 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.473 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.473 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:13:52.473 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:13:52.473 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.473 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:13:52.473 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6816", 2026-02-24T18:13:52.473 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:13:52.473 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.473 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.473 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:13:52.473 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6817", 2026-02-24T18:13:52.473 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:13:52.473 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.474 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.474 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.474 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:13:52.474 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:13:52.474 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.474 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:13:52.474 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6814", 2026-02-24T18:13:52.474 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:13:52.474 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.474 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.474 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:13:52.474 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6815", 2026-02-24T18:13:52.474 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:13:52.474 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.474 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.475 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.475 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6811/392529732", 2026-02-24T18:13:52.475 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6813/392529732", 2026-02-24T18:13:52.475 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6817/392529732", 2026-02-24T18:13:52.475 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6815/392529732", 2026-02-24T18:13:52.475 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:13:52.475 INFO:tasks.workunit.client.0.trial172.stdout: "exists" 2026-02-24T18:13:52.475 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.475 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.475 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.475 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:13:52.475 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "9c16fc6e-c787-4393-91f3-0b8a744ca5a0", 2026-02-24T18:13:52.475 INFO:tasks.workunit.client.0.trial172.stdout: "up": 1, 2026-02-24T18:13:52.475 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:13:52.475 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:13:52.476 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:13:52.476 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 15, 2026-02-24T18:13:52.476 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 30, 2026-02-24T18:13:52.476 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 39, 2026-02-24T18:13:52.476 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 0, 2026-02-24T18:13:52.476 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 33, 2026-02-24T18:13:52.476 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:13:52.476 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:13:52.476 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:13:52.476 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.476 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:13:52.476 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6810", 2026-02-24T18:13:52.476 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:13:52.476 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.476 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.477 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:13:52.477 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6811", 2026-02-24T18:13:52.477 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:13:52.477 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.477 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.477 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.477 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:13:52.477 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:13:52.477 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.477 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:13:52.477 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6812", 2026-02-24T18:13:52.477 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:13:52.477 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.477 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.477 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:13:52.478 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6813", 2026-02-24T18:13:52.478 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:13:52.478 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.478 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.478 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.478 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:13:52.478 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:13:52.478 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.478 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:13:52.478 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6816", 2026-02-24T18:13:52.478 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:13:52.478 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.478 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.478 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:13:52.478 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6817", 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6814", 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6815", 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.479 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6811/220259212", 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6813/220259212", 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6817/220259212", 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6815/220259212", 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: "exists", 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: "up" 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: "osd_xinfo": [ 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:13:17.570516+0000", 2026-02-24T18:13:52.480 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:12:41.784018+0000", 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:13:17.855892+0000", 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:12:45.212074+0000", 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:13:52.481 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:13:18.144390+0000", 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:12:49.415894+0000", 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap": [], 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap_items": [], 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap_primaries": [], 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: "pg_temp": [], 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: "primary_temp": [], 2026-02-24T18:13:52.482 INFO:tasks.workunit.client.0.trial172.stdout: "blocklist": {}, 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: "range_blocklist": {}, 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: "erasure_code_profiles": { 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: "default": { 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: "k": "2", 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: "m": "2", 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: "plugin": "isa", 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: "technique": "reed_sol_van" 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: "removed_snaps_queue": [], 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: "new_removed_snaps": [], 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: "new_purged_snaps": [], 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: "crush_node_flags": {}, 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: "device_class_flags": {}, 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode": { 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode_enabled": false, 2026-02-24T18:13:52.483 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_bucket_count": 0, 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: "degraded_stretch_mode": 0, 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: "recovering_stretch_mode": 0, 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode_bucket": 0 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: "osd_metadata": [ 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6802/266228372,v1:127.0.0.1:6803/266228372]", 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:13:52.484 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:13:52.485 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:13:52.486 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:13:52.486 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:12:38.941568Z", 2026-02-24T18:13:52.486 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:13:52.486 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:13:52.486 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:13:52.486 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:13:52.486 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:13:52.486 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:13:52.486 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:13:52.486 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6800/266228372,v1:127.0.0.1:6801/266228372]", 2026-02-24T18:13:52.486 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:13:52.486 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/266228372,v1:127.0.0.1:6807/266228372]", 2026-02-24T18:13:52.486 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/266228372,v1:127.0.0.1:6805/266228372]", 2026-02-24T18:13:52.486 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/0", 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:13:52.487 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6812/392529732,v1:127.0.0.1:6813/392529732]", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:13:52.488 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:12:42.772114Z", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6810/392529732,v1:127.0.0.1:6811/392529732]", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/392529732,v1:127.0.0.1:6817/392529732]", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/392529732,v1:127.0.0.1:6815/392529732]", 2026-02-24T18:13:52.489 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/1", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6812/220259212,v1:127.0.0.1:6813/220259212]", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:13:52.490 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:13:52.491 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:12:46.876457Z", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6810/220259212,v1:127.0.0.1:6811/220259212]", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/220259212,v1:127.0.0.1:6817/220259212]", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/220259212,v1:127.0.0.1:6815/220259212]", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:13:52.492 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/2", 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_clean_epochs": { 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "min_last_epoch_clean": 0, 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_clean": { 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "per_pool": [] 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "osd_epochs": [ 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 37 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 40 2026-02-24T18:13:52.493 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_first_committed": 1, 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_last_committed": 45, 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: "crushmap": { 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: "devices": [ 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.0", 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.1", 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.2", 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.494 INFO:tasks.workunit.client.0.trial172.stdout: "types": [ 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 0, 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd" 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: "name": "host" 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 2, 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: "name": "chassis" 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 3, 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: "name": "rack" 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 4, 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: "name": "row" 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.495 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.496 INFO:tasks.workunit.client.0.trial172.stderr:report 585435453 2026-02-24T18:13:52.496 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 5, 2026-02-24T18:13:52.496 INFO:tasks.workunit.client.0.trial172.stdout: "name": "pdu" 2026-02-24T18:13:52.496 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.496 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.496 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 6, 2026-02-24T18:13:52.496 INFO:tasks.workunit.client.0.trial172.stdout: "name": "pod" 2026-02-24T18:13:52.496 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.496 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.496 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 7, 2026-02-24T18:13:52.496 INFO:tasks.workunit.client.0.trial172.stdout: "name": "room" 2026-02-24T18:13:52.496 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.496 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.496 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 8, 2026-02-24T18:13:52.496 INFO:tasks.workunit.client.0.trial172.stdout: "name": "datacenter" 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 9, 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: "name": "zone" 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 10, 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: "name": "region" 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: "name": "root" 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: "buckets": [ 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: "id": -1, 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: "name": "default", 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "root", 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:13:52.497 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "id": -3, 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "id": -2, 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "name": "default~ssd", 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "root", 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "id": -4, 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.498 INFO:tasks.workunit.client.0.trial172.stdout: "id": -3, 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "name": "trial172", 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "host", 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 1 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 2 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.499 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "id": -4, 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "name": "trial172~ssd", 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "host", 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 1 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 2 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.500 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "rules": [ 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "rule_id": 0, 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "rule_name": "replicated_rule", 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "type": 1, 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "steps": [ 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "op": "take", 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "item": -1, 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "item_name": "default" 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "op": "choose_firstn", 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "num": 0, 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "type": "osd" 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "op": "emit" 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "tunables": { 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "choose_local_tries": 0, 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "choose_local_fallback_tries": 0, 2026-02-24T18:13:52.501 INFO:tasks.workunit.client.0.trial172.stdout: "choose_total_tries": 50, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_descend_once": 1, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_vary_r": 1, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_stable": 1, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "msr_descents": 100, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "msr_collision_tries": 100, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "straw_calc_version": 1, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "allowed_bucket_algs": 54, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "profile": "jewel", 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "optimal_tunables": 1, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "legacy_tunables": 0, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "minimum_required_version": "jewel", 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables": 1, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables2": 1, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "has_v2_rules": 0, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables3": 1, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "has_v3_rules": 0, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "has_v4_buckets": 1, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables5": 1, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "has_v5_rules": 0, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "has_msr_rules": 0 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "choose_args": {} 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "fsmap": { 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 1, 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "btime": "2026-02-24T18:12:37:946472+0000", 2026-02-24T18:13:52.502 INFO:tasks.workunit.client.0.trial172.stdout: "default_fscid": -1, 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "compat": { 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "compat": {}, 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "ro_compat": {}, 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "incompat": { 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_1": "base v0.20", 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_2": "client writeable ranges", 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_3": "default file layouts on dirs", 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_4": "dir inode in separate object", 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_5": "mds uses versioned encoding", 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_8": "no anchor table", 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_9": "file layout v2", 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_10": "snaprealm v2", 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_11": "minor log segments", 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_12": "quiesce subvolumes" 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "feature_flags": { 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "enable_multiple": true, 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "ever_enabled_multiple": true 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "standbys": [], 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "filesystems": [] 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "mdsmap_first_committed": 1, 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "mdsmap_last_committed": 1, 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "auth": { 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "first_committed": 1, 2026-02-24T18:13:52.503 INFO:tasks.workunit.client.0.trial172.stdout: "last_committed": 5, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_secrets": 3 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg": 32, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_active": 32, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_unknown": 0, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_osd": 3, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "pool_sum": { 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "stat_sum": { 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes": 51200, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects": 100, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_clones": 0, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_copies": 300, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing": 0, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_degraded": 100, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_misplaced": 0, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_unfound": 0, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_dirty": 100, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_whiteouts": 0, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_read": 0, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_read_kb": 0, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_write": 100, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_write_kb": 100, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_scrub_errors": 0, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_deep_scrub_errors": 0, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_recovered": 0, 2026-02-24T18:13:52.504 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_recovered": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_keys_recovered": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_omap": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_kb": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_kb": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_promote": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_high": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_low": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_some": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_full": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_pinned": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_legacy_snapsets": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_large_omap_objects": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_manifest": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_bytes": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_keys": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_repaired": 0 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "store_stats": { 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "total": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "available": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 0, 2026-02-24T18:13:52.505 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 0, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 0 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "log_size": 512, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "ondisk_log_size": 512, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "up": 64, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "acting": 64, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "num_store_stats": 0 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "osd_sum": { 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 0, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 0, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "num_pgs": 65, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "num_osds": 3, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_osds": 3, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_omap_osds": 3, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "kb": 314572800, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used": 83960, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_data": 2176, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_omap": 27, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_meta": 81700, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "kb_avail": 314488840, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "statfs": { 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "total": 322122547200, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "available": 322036572160, 2026-02-24T18:13:52.506 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 2228224, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 456546, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 28012, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 83661460 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "hb_peers": [], 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "snap_trim_queue_len": 0, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "num_snap_trimming": 0, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "num_shards_repaired": 0, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "op_queue_age_hist": { 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "histogram": [], 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "upper_bound": 1 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "perf_stat": { 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ms": 0, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ms": 0, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ns": 0, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ns": 0 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "alerts": [], 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "network_ping_times": [] 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "osd_sum_by_class": { 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "ssd": { 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 0, 2026-02-24T18:13:52.507 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 0, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "num_pgs": 65, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "num_osds": 3, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_osds": 3, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_omap_osds": 3, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "kb": 314572800, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used": 83960, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_data": 2176, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_omap": 27, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_meta": 81700, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "kb_avail": 314488840, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "statfs": { 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "total": 322122547200, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "available": 322036572160, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 2228224, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 456546, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 28012, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 83661460 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "hb_peers": [], 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "snap_trim_queue_len": 0, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "num_snap_trimming": 0, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "num_shards_repaired": 0, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "op_queue_age_hist": { 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "histogram": [], 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "upper_bound": 1 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "perf_stat": { 2026-02-24T18:13:52.508 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ms": 0, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ms": 0, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ns": 0, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ns": 0 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "alerts": [], 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "network_ping_times": [] 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "pool_stats": [ 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": 1, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg": 32, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "stat_sum": { 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes": 51200, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects": 100, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_clones": 0, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_copies": 300, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing": 0, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_degraded": 100, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_misplaced": 0, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_unfound": 0, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_dirty": 100, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_whiteouts": 0, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_read": 0, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_read_kb": 0, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_write": 100, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_write_kb": 100, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_scrub_errors": 0, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T18:13:52.509 INFO:tasks.workunit.client.0.trial172.stdout: "num_deep_scrub_errors": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_recovered": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_recovered": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_keys_recovered": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_omap": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_kb": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_kb": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_promote": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_high": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_low": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_some": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_full": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_pinned": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_legacy_snapsets": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_large_omap_objects": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_manifest": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_bytes": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_keys": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_repaired": 0 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "store_stats": { 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "total": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "available": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 1220608, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 153552, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:13:52.510 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 0, 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 0 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "log_size": 512, 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "ondisk_log_size": 512, 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "up": 64, 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "acting": 64, 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "num_store_stats": 3 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "osd_stats": [ 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 158913789961 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 42949672973 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 167503724552 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_by_state": [ 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "state": "active+undersized+degraded+wait", 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "num": 1 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "state": "active+undersized+degraded", 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "num": 31 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.511 INFO:tasks.workunit.client.0.trial172.stdout: "pool_pg_unavailable_map": [ 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": "1", 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "pgs": [] 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_by_osd": [ 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 32, 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 32, 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 32, 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 0, 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snaps": [ 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "pool": 1, 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snaps": {} 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:13:52.512 INFO:tasks.workunit.client.0.trial172.stdout: "servicemap": { 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 3, 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:13:46.422181+0000", 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "services": { 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "osd": { 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "daemons": { 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "summary": "", 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "0": { 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "start_epoch": 0, 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "start_stamp": "0.000000", 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "gid": 0, 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "metadata": {}, 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "task_status": {} 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "2": { 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "start_epoch": 0, 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "start_stamp": "0.000000", 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "gid": 0, 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "metadata": {}, 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "task_status": {} 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "mgrstat_first_committed": 1, 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "mgrstat_last_committed": 37, 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "logm_first_committed": 1, 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "logm_last_committed": 232, 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "paxos": { 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "first_committed": 1, 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "last_committed": 325, 2026-02-24T18:13:52.513 INFO:tasks.workunit.client.0.trial172.stdout: "last_pn": 0, 2026-02-24T18:13:52.514 INFO:tasks.workunit.client.0.trial172.stdout: "accepted_pn": 0 2026-02-24T18:13:52.514 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:13:52.514 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:13:52.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-24T18:13:52.514 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:755: TEST_divergent_3: expr 2 + 2 2026-02-24T18:13:52.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:755: TEST_divergent_3: objname=existing_4 2026-02-24T18:13:52.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:756: TEST_divergent_3: echo 'writing non-divergent object existing_4' 2026-02-24T18:13:52.514 INFO:tasks.workunit.client.0.trial172.stdout:writing non-divergent object existing_4 2026-02-24T18:13:52.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:757: TEST_divergent_3: ceph pg dump pgs 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.1f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586737+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.1e 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585701+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.1d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585464+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586731+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586791+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.9 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586985+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.8 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585828+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.7 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.588050+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.6 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587000+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.5 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587134+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.4 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586799+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.3 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585633+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.2 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586688+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585557+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.1 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585720+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.1c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.583668+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:45.190446+0000 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.0 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded+wait 2026-02-24T18:13:42.582973+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587174+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.e 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587167+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.657 INFO:tasks.workunit.client.0.trial172.stdout:1.f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586904+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.658 INFO:tasks.workunit.client.0.trial172.stdout:1.10 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584246+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.658 INFO:tasks.workunit.client.0.trial172.stdout:1.11 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584341+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.658 INFO:tasks.workunit.client.0.trial172.stdout:1.12 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584126+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.658 INFO:tasks.workunit.client.0.trial172.stdout:1.13 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.583984+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.658 INFO:tasks.workunit.client.0.trial172.stdout:1.14 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585448+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.658 INFO:tasks.workunit.client.0.trial172.stdout:1.15 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585754+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.658 INFO:tasks.workunit.client.0.trial172.stdout:1.16 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585346+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.658 INFO:tasks.workunit.client.0.trial172.stdout:1.17 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587317+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.658 INFO:tasks.workunit.client.0.trial172.stdout:1.18 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584089+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.658 INFO:tasks.workunit.client.0.trial172.stdout:1.19 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586770+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.658 INFO:tasks.workunit.client.0.trial172.stdout:1.1a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584783+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.658 INFO:tasks.workunit.client.0.trial172.stdout:1.1b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584576+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:13:52.658 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:13:52.658 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:13:52.658 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:13:52.665 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:758: TEST_divergent_3: rados -p test put existing_4 /tmp/tmp.9dNuV2JkPI 2026-02-24T18:13:52.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:760: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-24T18:13:52.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:760: TEST_divergent_3: wait_for_clean 2026-02-24T18:13:52.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:13:52.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:13:52.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:13:52.697 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-24T18:13:52.697 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:13:52.698 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:13:52.698 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:13:52.698 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:13:52.698 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:13:52.758 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-24T18:13:52.758 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:13:52.759 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:13:52.759 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:13:52.759 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:13:52.759 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:13:52.974 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:13:52.975 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:13:52.975 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:13:52.975 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:13:52.975 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:13:52.975 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:13:53.057 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=158913789963 2026-02-24T18:13:53.057 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 158913789963 2026-02-24T18:13:53.057 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-158913789963' 2026-02-24T18:13:53.057 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:13:53.057 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:13:53.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq= 2026-02-24T18:13:53.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: true 2026-02-24T18:13:53.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z '' 2026-02-24T18:13:53.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: flush_pg_stats: continue 2026-02-24T18:13:53.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:13:53.125 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:13:53.206 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=167503724555 2026-02-24T18:13:53.206 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 167503724555 2026-02-24T18:13:53.206 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-158913789963 2-167503724555' 2026-02-24T18:13:53.206 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:13:53.207 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-158913789963 2026-02-24T18:13:53.207 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:13:53.207 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:13:53.208 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-158913789963 2026-02-24T18:13:53.208 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:13:53.210 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=158913789963 2026-02-24T18:13:53.210 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 158913789963 2026-02-24T18:13:53.210 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 158913789963' 2026-02-24T18:13:53.211 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:13:53.438 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789961 -lt 158913789963 2026-02-24T18:13:53.438 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:13:54.439 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:13:54.439 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:13:54.661 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789963 -lt 158913789963 2026-02-24T18:13:54.661 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:13:54.662 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-167503724555 2026-02-24T18:13:54.662 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:13:54.663 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:13:54.664 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-167503724555 2026-02-24T18:13:54.664 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:13:54.665 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=167503724555 2026-02-24T18:13:54.665 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 167503724555 2026-02-24T18:13:54.666 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 167503724555' 2026-02-24T18:13:54.666 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:13:54.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 167503724555 -lt 167503724555 2026-02-24T18:13:54.886 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:13:54.886 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:54.886 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:55.175 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-24T18:13:55.176 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:13:55.176 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:13:55.176 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:13:55.176 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:13:55.176 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:13:55.176 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:13:55.177 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:13:55.393 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:13:55.394 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:13:55.394 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:55.394 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:55.681 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-24T18:13:55.681 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-24T18:13:55.681 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-24T18:13:55.681 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-24T18:13:55.681 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:13:55.682 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-24T18:13:55.782 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:13:55.782 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:13:55.783 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:13:55.783 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:13:55.783 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:13:55.783 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:13:55.783 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:13:55.784 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:13:55.995 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:13:55.996 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:13:55.996 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:55.996 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:56.274 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-24T18:13:56.274 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:13:56.274 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:13:56.274 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:13:56.274 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:13:56.274 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:13:56.274 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:13:56.275 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:13:56.275 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:13:56.560 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:13:56.561 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:13:56.561 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 8 )) 2026-02-24T18:13:56.561 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:13:56.561 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-24T18:13:56.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:13:56.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:13:56.762 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:13:56.762 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:13:56.762 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:13:56.762 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:13:56.762 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:13:56.762 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:13:56.986 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:13:56.987 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:13:56.987 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:56.987 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:57.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-24T18:13:57.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:13:57.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:13:57.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:13:57.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:13:57.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:13:57.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:13:57.285 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:13:57.285 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:13:57.572 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:13:57.573 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:13:57.573 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 8 )) 2026-02-24T18:13:57.573 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:13:57.573 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-24T18:13:57.973 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:13:57.973 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:13:57.974 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:13:57.974 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:13:57.975 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:13:57.975 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:13:57.975 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:13:57.975 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:13:58.194 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:13:58.195 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:13:58.195 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:58.196 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:13:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-24T18:13:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:13:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:13:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:13:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:13:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:13:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:13:58.484 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:13:58.484 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:13:58.773 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:13:58.773 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:13:58.773 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 8 )) 2026-02-24T18:13:58.773 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:13:58.773 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-24T18:13:59.573 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:13:59.573 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:13:59.574 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:13:59.575 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:13:59.575 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:13:59.575 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:13:59.575 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:13:59.575 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:13:59.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:13:59.788 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:13:59.788 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:13:59.788 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:14:00.081 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-24T18:14:00.081 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:14:00.081 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:14:00.081 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:14:00.081 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:14:00.081 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:14:00.082 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:14:00.082 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:14:00.082 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:14:00.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:14:00.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:14:00.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 8 )) 2026-02-24T18:14:00.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:14:00.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 1.6 2026-02-24T18:14:01.974 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:14:01.974 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:14:01.974 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:14:01.974 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:14:01.974 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:14:01.974 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:14:01.975 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:14:01.975 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:14:02.189 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:14:02.190 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:14:02.191 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:14:02.191 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:14:02.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-24T18:14:02.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:14:02.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:14:02.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:14:02.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:14:02.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:14:02.476 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:14:02.476 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:14:02.476 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:14:02.757 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:14:02.757 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:14:02.758 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 5 >= 8 )) 2026-02-24T18:14:02.758 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:14:02.758 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 3.2 2026-02-24T18:14:05.957 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:14:05.957 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:14:05.958 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:14:05.958 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:14:05.958 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:14:05.958 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:14:05.958 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:14:05.958 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:14:06.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:14:06.186 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:14:06.186 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:14:06.187 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:14:06.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-24T18:14:06.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:14:06.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:14:06.482 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:14:06.482 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:14:06.482 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:14:06.482 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:14:06.482 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:14:06.482 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:14:06.765 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:14:06.765 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:14:06.765 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 6 >= 8 )) 2026-02-24T18:14:06.765 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:14:06.765 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 6.4 2026-02-24T18:14:13.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:14:13.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:14:13.167 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:14:13.168 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:14:13.168 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:14:13.168 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:14:13.168 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:14:13.168 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:14:13.389 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:14:13.389 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:14:13.389 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:14:13.390 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:14:13.665 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-24T18:14:13.665 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:14:13.665 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:14:13.665 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:14:13.665 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:14:13.665 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:14:13.665 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:14:13.665 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:14:13.665 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:14:13.948 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:14:13.949 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:14:13.949 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 7 >= 8 )) 2026-02-24T18:14:13.949 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:14:13.949 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 7.3 2026-02-24T18:14:21.249 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:14:21.250 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:14:21.250 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:14:21.250 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:14:21.250 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:14:21.250 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:14:21.250 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:14:21.251 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:14:21.466 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:14:21.466 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:14:21.467 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:14:21.467 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:14:21.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-24T18:14:21.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:14:21.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:14:21.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:14:21.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:14:21.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:14:21.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:14:21.752 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:14:21.753 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:14:22.041 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:14:22.042 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:14:22.042 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 8 >= 8 )) 2026-02-24T18:14:22.042 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-24T18:14:22.323 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:14:22.323 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_fingerprint": "09dc039b-8fcf-4bae-aa5e-951070c69b4f", 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "version": "20.3.0-5507-g618c318a", 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "commit": "618c318a7a0b287ab5aeff207b7bf1d73ef45044", 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "timestamp": "2026-02-24T18:14:22.321869+0000", 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "tag": "", 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "health": { 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "status": "HEALTH_WARN", 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "checks": { 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "OSDMAP_FLAGS": { 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "count": 17 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [], 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "OSD_DOWN": { 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "message": "1 osds down", 2026-02-24T18:14:22.324 INFO:tasks.workunit.client.0.trial172.stdout: "count": 1 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [ 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: "message": "osd.1 (root=default,host=trial172) is down" 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: "PG_DEGRADED": { 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 32 pgs degraded", 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: "count": 32 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [ 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.1 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.325 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.2 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.3 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.4 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.5 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.6 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.7 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.8 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.9 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.326 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.a is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.b is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.c is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.d is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.e is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.f is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.10 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.11 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.327 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.12 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.13 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.14 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.15 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.16 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.17 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.18 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.19 is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.1a is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.328 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.1b is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.1c is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.1d is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.1e is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.1f is active+undersized+degraded, acting [0,2]" 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: "mutes": [] 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: "monmap_first_committed": 1, 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: "monmap_last_committed": 1, 2026-02-24T18:14:22.329 INFO:tasks.workunit.client.0.trial172.stdout: "monmap": { 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 1, 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "fsid": "27512777-e37f-4f16-b176-1621842bd4df", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:12:37.858142Z", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "created": "2026-02-24T18:12:37.858142Z", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "min_mon_release": 20, 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "min_mon_release_name": "tentacle", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "election_strategy": 1, 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "disallowed_leaders": "", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode": false, 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "tiebreaker_mon": "", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "removed_ranks": "", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "features": { 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "persistent": [ 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "kraken", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "luminous", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "mimic", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap-prune", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "nautilus", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "octopus", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "pacific", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "elector-pinging", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "quincy", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "reef", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "squid", 2026-02-24T18:14:22.330 INFO:tasks.workunit.client.0.trial172.stdout: "tentacle", 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "nvmeof_beacon_diff" 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "optional": [] 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "mons": [ 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "rank": 0, 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "name": "a", 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:7115", 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 0 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:7115/0", 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "priority": 0, 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 0, 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "time_added": "2026-02-24T18:12:37:858137+0000", 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: "crush_location": [] 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.331 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "quorum": [ 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: 0 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap": { 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 45, 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "fsid": "27512777-e37f-4f16-b176-1621842bd4df", 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "created": "2026-02-24T18:12:37.946691+0000", 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:13:42.573809+0000", 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "last_up_change": "2026-02-24T18:13:21.902285+0000", 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "last_in_change": "2026-02-24T18:13:22.809612+0000", 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "flags_num": 5800384, 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "flags_set": [ 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "nodown", 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "noin", 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "noout", 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "pglog_hardlimit", 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snapdirs", 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "recovery_deletes", 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "sortbitwise" 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "crush_version": 9, 2026-02-24T18:14:22.332 INFO:tasks.workunit.client.0.trial172.stdout: "full_ratio": 0.99000000953674316, 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_snapshot": "", 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "pool_max": 1, 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "max_osd": 3, 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "require_min_compat_client": "luminous", 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "min_compat_client": "jewel", 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "require_osd_release": "tentacle", 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "allow_crimson": false, 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "pools": [ 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "pool": 1, 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "pool_name": "test", 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "create_time": "2026-02-24T18:12:51.223307+0000", 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "flags": 1, 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "flags_names": "hashpspool", 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "type": 1, 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "size": 3, 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "min_size": 2, 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "crush_rule": 0, 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_count": 0, 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_target": 0, 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_barrier": 0, 2026-02-24T18:14:22.333 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "is_stretch_pool": false, 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "object_hash": 2, 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "pg_autoscale_mode": "on", 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num": 32, 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "pg_placement_num": 1, 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "pg_placement_num_target": 32, 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num_target": 32, 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num_pending": 32, 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "last_pg_merge_meta": { 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "source_pgid": "0.0", 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "ready_epoch": 0, 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_started": 0, 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_clean": 0, 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "source_version": "0'0", 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "target_version": "0'0" 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "last_change": "44", 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend": "0", 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend_preluminous": "44", 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "auid": 0, 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "snap_mode": "selfmanaged", 2026-02-24T18:14:22.334 INFO:tasks.workunit.client.0.trial172.stdout: "snap_seq": 0, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "snap_epoch": 0, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "pool_snaps": [], 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "removed_snaps": "[]", 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "quota_max_bytes": 0, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "quota_max_objects": 0, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "tiers": [], 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "tier_of": -1, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "read_tier": -1, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "write_tier": -1, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "cache_mode": "none", 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "target_max_bytes": 0, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "target_max_objects": 0, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "cache_min_flush_age": 0, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "cache_min_evict_age": 0, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "erasure_code_profile": "", 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_params": { 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "type": "none" 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_period": 0, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_count": 0, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "use_gmt_hitset": true, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "min_read_recency_for_promote": 0, 2026-02-24T18:14:22.335 INFO:tasks.workunit.client.0.trial172.stdout: "min_write_recency_for_promote": 0, 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_grade_decay_rate": 0, 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_search_last_n": 0, 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "grade_table": [], 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "stripe_width": 0, 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "expected_num_objects": 0, 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "fast_read": false, 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "nonprimary_shards": "{}", 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "options": {}, 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "application_metadata": {}, 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "read_balance": { 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "score_type": "Fair distribution", 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "score_acting": 2.9900000095367432, 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "score_stable": 2.9900000095367432, 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "optimal_score": 0.67000001668930054, 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "raw_score_acting": 2, 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "raw_score_stable": 2, 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity_weighted": 1, 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "average_primary_affinity": 1, 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "average_primary_affinity_weighted": 1 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "osds": [ 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.336 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "1b2deac1-b721-42fd-ace2-60433262abf1", 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "up": 1, 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 5, 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 30, 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 37, 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 44, 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 31, 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6800", 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6801", 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.337 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6802", 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6803", 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6806", 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:14:22.338 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6807", 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6804", 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6805", 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 266228372 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6801/266228372", 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6803/266228372", 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6807/266228372", 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6805/266228372", 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "exists", 2026-02-24T18:14:22.339 INFO:tasks.workunit.client.0.trial172.stdout: "up" 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "43112d05-9157-407e-ba40-33f71a9b44d7", 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "up": 0, 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 0, 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 0, 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 10, 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 22, 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 32, 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6810", 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:14:22.340 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6811", 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6812", 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6813", 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6816", 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:14:22.341 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6817", 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6814", 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6815", 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 392529732 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6811/392529732", 2026-02-24T18:14:22.342 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6813/392529732", 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6817/392529732", 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6815/392529732", 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "exists" 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "9c16fc6e-c787-4393-91f3-0b8a744ca5a0", 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "up": 1, 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 15, 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 30, 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 39, 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 0, 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 33, 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6810", 2026-02-24T18:14:22.343 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6811", 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6812", 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6813", 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:14:22.344 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6816", 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6817", 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6814", 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6815", 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 220259212 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.345 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6811/220259212", 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6813/220259212", 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6817/220259212", 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6815/220259212", 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "exists", 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "up" 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "osd_xinfo": [ 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:13:17.570516+0000", 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:12:41.784018+0000", 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:14:22.346 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:13:17.855892+0000", 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:12:45.212074+0000", 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:13:18.144390+0000", 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:12:49.415894+0000", 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap": [], 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap_items": [], 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap_primaries": [], 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "pg_temp": [], 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "primary_temp": [], 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "blocklist": {}, 2026-02-24T18:14:22.347 INFO:tasks.workunit.client.0.trial172.stdout: "range_blocklist": {}, 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "erasure_code_profiles": { 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "default": { 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "k": "2", 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "m": "2", 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "plugin": "isa", 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "technique": "reed_sol_van" 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "removed_snaps_queue": [], 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "new_removed_snaps": [], 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "new_purged_snaps": [], 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "crush_node_flags": {}, 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "device_class_flags": {}, 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode": { 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode_enabled": false, 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_bucket_count": 0, 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "degraded_stretch_mode": 0, 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "recovering_stretch_mode": 0, 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode_bucket": 0 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "osd_metadata": [ 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:14:22.348 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6802/266228372,v1:127.0.0.1:6803/266228372]", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:14:22.349 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:12:38.941568Z", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6800/266228372,v1:127.0.0.1:6801/266228372]", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/266228372,v1:127.0.0.1:6807/266228372]", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/266228372,v1:127.0.0.1:6805/266228372]", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/0", 2026-02-24T18:14:22.350 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6812/392529732,v1:127.0.0.1:6813/392529732]", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:14:22.351 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:12:42.772114Z", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6810/392529732,v1:127.0.0.1:6811/392529732]", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/392529732,v1:127.0.0.1:6817/392529732]", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/392529732,v1:127.0.0.1:6815/392529732]", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:14:22.352 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/1", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6812/220259212,v1:127.0.0.1:6813/220259212]", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:14:22.353 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:12:46.876457Z", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:14:22.354 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6810/220259212,v1:127.0.0.1:6811/220259212]", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/220259212,v1:127.0.0.1:6817/220259212]", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/220259212,v1:127.0.0.1:6815/220259212]", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/2", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_clean_epochs": { 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "min_last_epoch_clean": 0, 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_clean": { 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: "per_pool": [] 2026-02-24T18:14:22.355 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: "osd_epochs": [ 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 37 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 40 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_first_committed": 1, 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_last_committed": 45, 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: "crushmap": { 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: "devices": [ 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.0", 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.1", 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:14:22.356 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.2", 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: "types": [ 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 0, 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd" 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: "name": "host" 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 2, 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: "name": "chassis" 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 3, 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: "name": "rack" 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 4, 2026-02-24T18:14:22.357 INFO:tasks.workunit.client.0.trial172.stdout: "name": "row" 2026-02-24T18:14:22.358 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.358 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.358 INFO:tasks.workunit.client.0.trial172.stderr:report 2680599714 2026-02-24T18:14:22.358 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 5, 2026-02-24T18:14:22.358 INFO:tasks.workunit.client.0.trial172.stdout: "name": "pdu" 2026-02-24T18:14:22.358 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.358 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.358 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 6, 2026-02-24T18:14:22.358 INFO:tasks.workunit.client.0.trial172.stdout: "name": "pod" 2026-02-24T18:14:22.358 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.358 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 7, 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: "name": "room" 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 8, 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: "name": "datacenter" 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 9, 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: "name": "zone" 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 10, 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: "name": "region" 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: "name": "root" 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: "buckets": [ 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: "id": -1, 2026-02-24T18:14:22.359 INFO:tasks.workunit.client.0.trial172.stdout: "name": "default", 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "root", 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "id": -3, 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "id": -2, 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "name": "default~ssd", 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "root", 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.360 INFO:tasks.workunit.client.0.trial172.stdout: "id": -4, 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "id": -3, 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "name": "trial172", 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "host", 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 1 2026-02-24T18:14:22.361 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 2 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "id": -4, 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "name": "trial172~ssd", 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "host", 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 1 2026-02-24T18:14:22.362 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 2 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: "rules": [ 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: "rule_id": 0, 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: "rule_name": "replicated_rule", 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: "type": 1, 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: "steps": [ 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: "op": "take", 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: "item": -1, 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: "item_name": "default" 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: "op": "choose_firstn", 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: "num": 0, 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: "type": "osd" 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.363 INFO:tasks.workunit.client.0.trial172.stdout: "op": "emit" 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "tunables": { 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "choose_local_tries": 0, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "choose_local_fallback_tries": 0, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "choose_total_tries": 50, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_descend_once": 1, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_vary_r": 1, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_stable": 1, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "msr_descents": 100, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "msr_collision_tries": 100, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "straw_calc_version": 1, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "allowed_bucket_algs": 54, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "profile": "jewel", 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "optimal_tunables": 1, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "legacy_tunables": 0, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "minimum_required_version": "jewel", 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables": 1, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables2": 1, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "has_v2_rules": 0, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables3": 1, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "has_v3_rules": 0, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "has_v4_buckets": 1, 2026-02-24T18:14:22.364 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables5": 1, 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "has_v5_rules": 0, 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "has_msr_rules": 0 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "choose_args": {} 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "fsmap": { 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 1, 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "btime": "2026-02-24T18:12:37:946472+0000", 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "default_fscid": -1, 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "compat": { 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "compat": {}, 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "ro_compat": {}, 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "incompat": { 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "feature_1": "base v0.20", 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "feature_2": "client writeable ranges", 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "feature_3": "default file layouts on dirs", 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "feature_4": "dir inode in separate object", 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "feature_5": "mds uses versioned encoding", 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "feature_8": "no anchor table", 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "feature_9": "file layout v2", 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "feature_10": "snaprealm v2", 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "feature_11": "minor log segments", 2026-02-24T18:14:22.365 INFO:tasks.workunit.client.0.trial172.stdout: "feature_12": "quiesce subvolumes" 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "feature_flags": { 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "enable_multiple": true, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "ever_enabled_multiple": true 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "standbys": [], 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "filesystems": [] 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "mdsmap_first_committed": 1, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "mdsmap_last_committed": 1, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "auth": { 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "first_committed": 1, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "last_committed": 5, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "num_secrets": 3 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg": 32, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_active": 32, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_unknown": 0, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "num_osd": 3, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "pool_sum": { 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "stat_sum": { 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes": 51688, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects": 100, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_clones": 0, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_copies": 300, 2026-02-24T18:14:22.366 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_degraded": 100, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_misplaced": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_unfound": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_dirty": 100, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_whiteouts": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_read": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_read_kb": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_write": 101, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_write_kb": 101, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_scrub_errors": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_deep_scrub_errors": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_recovered": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_recovered": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_keys_recovered": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_omap": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_kb": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_kb": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_promote": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_high": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_low": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_some": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_full": 0, 2026-02-24T18:14:22.367 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_pinned": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "num_legacy_snapsets": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "num_large_omap_objects": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_manifest": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_bytes": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_keys": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_repaired": 0 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "store_stats": { 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "total": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "available": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 0 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "log_size": 513, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "ondisk_log_size": 513, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "up": 64, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "acting": 64, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "num_store_stats": 0 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "osd_sum": { 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 0, 2026-02-24T18:14:22.368 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 0, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "num_pgs": 65, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "num_osds": 3, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_osds": 3, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_omap_osds": 3, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "kb": 314572800, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used": 83960, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_data": 2176, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_omap": 27, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_meta": 81700, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "kb_avail": 314488840, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "statfs": { 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "total": 322122547200, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "available": 322036572160, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 2228224, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 457522, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 28012, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 83661460 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "hb_peers": [], 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "snap_trim_queue_len": 0, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "num_snap_trimming": 0, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "num_shards_repaired": 0, 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "op_queue_age_hist": { 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "histogram": [], 2026-02-24T18:14:22.369 INFO:tasks.workunit.client.0.trial172.stdout: "upper_bound": 1 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "perf_stat": { 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ms": 0, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ms": 0, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ns": 0, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ns": 0 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "alerts": [], 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "network_ping_times": [] 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "osd_sum_by_class": { 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "ssd": { 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 0, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 0, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "num_pgs": 65, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "num_osds": 3, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_osds": 3, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_omap_osds": 3, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "kb": 314572800, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used": 83960, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_data": 2176, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_omap": 27, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_meta": 81700, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "kb_avail": 314488840, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "statfs": { 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "total": 322122547200, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "available": 322036572160, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 2228224, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 457522, 2026-02-24T18:14:22.370 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 28012, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 83661460 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "hb_peers": [], 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "snap_trim_queue_len": 0, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "num_snap_trimming": 0, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "num_shards_repaired": 0, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "op_queue_age_hist": { 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "histogram": [], 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "upper_bound": 1 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "perf_stat": { 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ms": 0, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ms": 0, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ns": 0, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ns": 0 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "alerts": [], 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "network_ping_times": [] 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "pool_stats": [ 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": 1, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg": 32, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "stat_sum": { 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes": 51688, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects": 100, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_clones": 0, 2026-02-24T18:14:22.371 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_copies": 300, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_degraded": 100, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_misplaced": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_unfound": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_dirty": 100, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_whiteouts": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_read": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_read_kb": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_write": 101, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_write_kb": 101, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_scrub_errors": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_deep_scrub_errors": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_recovered": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_recovered": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_keys_recovered": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_omap": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_kb": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_kb": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_promote": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_high": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_low": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_some": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_full": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_pinned": 0, 2026-02-24T18:14:22.372 INFO:tasks.workunit.client.0.trial172.stdout: "num_legacy_snapsets": 0, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "num_large_omap_objects": 0, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_manifest": 0, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_bytes": 0, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_keys": 0, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_repaired": 0 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "store_stats": { 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "total": 0, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "available": 0, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 1220608, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 154528, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 0, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 0 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "log_size": 513, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "ondisk_log_size": 513, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "up": 64, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "acting": 64, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "num_store_stats": 3 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "osd_stats": [ 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 158913789969 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.373 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 42949672973 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 167503724560 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_by_state": [ 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "state": "active+undersized+degraded", 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "num": 32 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "pool_pg_unavailable_map": [ 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": "1", 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "pgs": [] 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_by_osd": [ 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 32, 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 32, 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 32, 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.374 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 0, 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snaps": [ 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "pool": 1, 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snaps": {} 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "servicemap": { 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 4, 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:14:20.427943+0000", 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "services": { 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "osd": { 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "daemons": { 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "summary": "", 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "2": { 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "start_epoch": 0, 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "start_stamp": "0.000000", 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "gid": 0, 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "metadata": {}, 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "task_status": {} 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "mgrstat_first_committed": 1, 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "mgrstat_last_committed": 52, 2026-02-24T18:14:22.375 INFO:tasks.workunit.client.0.trial172.stdout: "logm_first_committed": 1, 2026-02-24T18:14:22.376 INFO:tasks.workunit.client.0.trial172.stdout: "logm_last_committed": 297, 2026-02-24T18:14:22.376 INFO:tasks.workunit.client.0.trial172.stdout: "paxos": { 2026-02-24T18:14:22.376 INFO:tasks.workunit.client.0.trial172.stdout: "first_committed": 1, 2026-02-24T18:14:22.376 INFO:tasks.workunit.client.0.trial172.stdout: "last_committed": 405, 2026-02-24T18:14:22.376 INFO:tasks.workunit.client.0.trial172.stdout: "last_pn": 0, 2026-02-24T18:14:22.376 INFO:tasks.workunit.client.0.trial172.stdout: "accepted_pn": 0 2026-02-24T18:14:22.376 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:14:22.376 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:14:22.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-24T18:14:22.376 INFO:tasks.workunit.client.0.trial172.stdout:delay recovery 2026-02-24T18:14:22.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:763: TEST_divergent_3: echo 'delay recovery' 2026-02-24T18:14:22.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:764: TEST_divergent_3: ceph pg dump pgs 2026-02-24T18:14:22.543 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:14:22.543 INFO:tasks.workunit.client.0.trial172.stdout:1.1f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586737+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.544 INFO:tasks.workunit.client.0.trial172.stdout:1.1e 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585701+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.544 INFO:tasks.workunit.client.0.trial172.stdout:1.1d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585464+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.544 INFO:tasks.workunit.client.0.trial172.stdout:1.b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586731+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.544 INFO:tasks.workunit.client.0.trial172.stdout:1.a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586791+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.544 INFO:tasks.workunit.client.0.trial172.stdout:1.9 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586985+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.544 INFO:tasks.workunit.client.0.trial172.stdout:1.8 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585828+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.544 INFO:tasks.workunit.client.0.trial172.stdout:1.7 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.588050+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.544 INFO:tasks.workunit.client.0.trial172.stdout:1.6 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587000+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.544 INFO:tasks.workunit.client.0.trial172.stdout:1.5 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587134+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.544 INFO:tasks.workunit.client.0.trial172.stdout:1.4 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586799+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.545 INFO:tasks.workunit.client.0.trial172.stdout:1.3 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585633+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.545 INFO:tasks.workunit.client.0.trial172.stdout:1.2 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586688+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.545 INFO:tasks.workunit.client.0.trial172.stdout:1.c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585557+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.545 INFO:tasks.workunit.client.0.trial172.stdout:1.1 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585720+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.545 INFO:tasks.workunit.client.0.trial172.stdout:1.1c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.583668+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:45.190446+0000 0 0 2026-02-24T18:14:22.545 INFO:tasks.workunit.client.0.trial172.stdout:1.0 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:54.908303+0000 30'100 45:144 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:44.199233+0000 0 0 2026-02-24T18:14:22.545 INFO:tasks.workunit.client.0.trial172.stdout:1.d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587174+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.545 INFO:tasks.workunit.client.0.trial172.stdout:1.e 3 0 3 0 0 2088 0 0 17 84 17 active+undersized+degraded 2026-02-24T18:13:42.587167+0000 45'101 45:144 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.545 INFO:tasks.workunit.client.0.trial172.stdout:1.f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586904+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.545 INFO:tasks.workunit.client.0.trial172.stdout:1.10 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584246+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.545 INFO:tasks.workunit.client.0.trial172.stdout:1.11 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584341+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.545 INFO:tasks.workunit.client.0.trial172.stdout:1.12 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584126+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.546 INFO:tasks.workunit.client.0.trial172.stdout:1.13 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.583984+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.546 INFO:tasks.workunit.client.0.trial172.stdout:1.14 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585448+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.546 INFO:tasks.workunit.client.0.trial172.stdout:1.15 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585754+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.546 INFO:tasks.workunit.client.0.trial172.stdout:1.16 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585346+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.546 INFO:tasks.workunit.client.0.trial172.stdout:1.17 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587317+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.546 INFO:tasks.workunit.client.0.trial172.stdout:1.18 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584089+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.546 INFO:tasks.workunit.client.0.trial172.stdout:1.19 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586770+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.546 INFO:tasks.workunit.client.0.trial172.stdout:1.1a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584783+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.546 INFO:tasks.workunit.client.0.trial172.stdout:1.1b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584576+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.546 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:14:22.546 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:14:22.547 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:14:22.554 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: for i in $non_divergent 2026-02-24T18:14:22.555 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:767: TEST_divergent_3: get_asok_path osd.0 2026-02-24T18:14:22.555 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T18:14:22.555 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T18:14:22.555 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:14:22.555 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:14:22.556 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:14:22.556 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.0.asok 2026-02-24T18:14:22.556 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:767: TEST_divergent_3: CEPH_ARGS= 2026-02-24T18:14:22.556 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:767: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.0.asok set_recovery_delay 100000 2026-02-24T18:14:22.622 INFO:tasks.workunit.client.0.trial172.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-24T18:14:22.628 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: for i in $non_divergent 2026-02-24T18:14:22.629 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:767: TEST_divergent_3: get_asok_path osd.2 2026-02-24T18:14:22.629 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-24T18:14:22.629 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-24T18:14:22.629 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:14:22.629 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:14:22.629 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:14:22.629 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.2.asok 2026-02-24T18:14:22.630 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:767: TEST_divergent_3: CEPH_ARGS= 2026-02-24T18:14:22.630 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:767: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.2.asok set_recovery_delay 100000 2026-02-24T18:14:22.684 INFO:tasks.workunit.client.0.trial172.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-24T18:14:22.689 INFO:tasks.workunit.client.0.trial172.stdout:revive divergent 1 2026-02-24T18:14:22.689 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:771: TEST_divergent_3: echo 'revive divergent 1' 2026-02-24T18:14:22.690 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:772: TEST_divergent_3: ceph pg dump pgs 2026-02-24T18:14:22.909 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:14:22.910 INFO:tasks.workunit.client.0.trial172.stdout:1.1f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586737+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.910 INFO:tasks.workunit.client.0.trial172.stdout:1.1e 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585701+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.910 INFO:tasks.workunit.client.0.trial172.stdout:1.1d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585464+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.910 INFO:tasks.workunit.client.0.trial172.stdout:1.b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586731+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.910 INFO:tasks.workunit.client.0.trial172.stdout:1.a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586791+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.910 INFO:tasks.workunit.client.0.trial172.stdout:1.9 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586985+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.910 INFO:tasks.workunit.client.0.trial172.stdout:1.8 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585828+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.910 INFO:tasks.workunit.client.0.trial172.stdout:1.7 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.588050+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.911 INFO:tasks.workunit.client.0.trial172.stdout:1.6 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587000+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.911 INFO:tasks.workunit.client.0.trial172.stdout:1.5 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587134+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.911 INFO:tasks.workunit.client.0.trial172.stdout:1.4 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586799+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.911 INFO:tasks.workunit.client.0.trial172.stdout:1.3 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585633+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.911 INFO:tasks.workunit.client.0.trial172.stdout:1.2 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586688+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.911 INFO:tasks.workunit.client.0.trial172.stdout:1.c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585557+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.911 INFO:tasks.workunit.client.0.trial172.stdout:1.1 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585720+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.911 INFO:tasks.workunit.client.0.trial172.stdout:1.1c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.583668+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:45.190446+0000 0 0 2026-02-24T18:14:22.911 INFO:tasks.workunit.client.0.trial172.stdout:1.0 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:54.908303+0000 30'100 45:144 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:44.199233+0000 0 0 2026-02-24T18:14:22.911 INFO:tasks.workunit.client.0.trial172.stdout:1.d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587174+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.911 INFO:tasks.workunit.client.0.trial172.stdout:1.e 3 0 3 0 0 2088 0 0 17 84 17 active+undersized+degraded 2026-02-24T18:13:42.587167+0000 45'101 45:144 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.911 INFO:tasks.workunit.client.0.trial172.stdout:1.f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586904+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.912 INFO:tasks.workunit.client.0.trial172.stdout:1.10 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584246+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.912 INFO:tasks.workunit.client.0.trial172.stdout:1.11 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584341+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.912 INFO:tasks.workunit.client.0.trial172.stdout:1.12 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584126+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.912 INFO:tasks.workunit.client.0.trial172.stdout:1.13 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.583984+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.912 INFO:tasks.workunit.client.0.trial172.stdout:1.14 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585448+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.912 INFO:tasks.workunit.client.0.trial172.stdout:1.15 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585754+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.912 INFO:tasks.workunit.client.0.trial172.stdout:1.16 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585346+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.912 INFO:tasks.workunit.client.0.trial172.stdout:1.17 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587317+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.912 INFO:tasks.workunit.client.0.trial172.stdout:1.18 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584089+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.912 INFO:tasks.workunit.client.0.trial172.stdout:1.19 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586770+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.912 INFO:tasks.workunit.client.0.trial172.stdout:1.1a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584783+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.912 INFO:tasks.workunit.client.0.trial172.stdout:1.1b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584576+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:22.913 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:14:22.913 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:14:22.913 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:14:22.920 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:773: TEST_divergent_3: ceph osd set noup 2026-02-24T18:14:23.239 INFO:tasks.workunit.client.0.trial172.stderr:noup is set 2026-02-24T18:14:23.255 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:774: TEST_divergent_3: activate_osd td/divergent-priors 1 2026-02-24T18:14:23.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:14:23.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:14:23.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T18:14:23.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:14:23.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-24T18:14:23.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:14:23.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:14:23.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:14:23.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:14:23.256 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-24T18:14:23.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-24T18:14:23.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:14:23.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:14:23.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:14:23.257 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:14:23.257 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:14:23.257 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:14:23.257 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:14:23.257 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:14:23.257 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:14:23.257 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:14:23.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:14:23.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:14:23.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:14:23.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:14:23.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:14:23.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:14:23.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:14:23.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:14:23.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:14:23.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:14:23.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-24T18:14:23.259 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:14:23.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T18:14:23.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:14:23.260 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-24T18:14:23.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T18:14:23.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:14:23.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:14:23.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:14:23.283 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:14:23.280+0000 7f35700d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:14:23.284 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:14:23.280+0000 7f35700d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:14:23.285 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:14:23.280+0000 7f35700d7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:14:23.480 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: return 0 2026-02-24T18:14:23.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:775: TEST_divergent_3: sleep 5 2026-02-24T18:14:23.656 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:14:23.652+0000 7f35700d7a40 -1 Falling back to public interface 2026-02-24T18:14:24.223 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:14:24.220+0000 7f35700d7a40 -1 osd.1 30 log_to_monitors true 2026-02-24T18:14:28.482 INFO:tasks.workunit.client.0.trial172.stdout:delay recovery divergent 2026-02-24T18:14:28.482 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:777: TEST_divergent_3: echo 'delay recovery divergent' 2026-02-24T18:14:28.482 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:778: TEST_divergent_3: ceph pg dump pgs 2026-02-24T18:14:28.686 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:14:28.686 INFO:tasks.workunit.client.0.trial172.stdout:1.1f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586737+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.686 INFO:tasks.workunit.client.0.trial172.stdout:1.1e 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585701+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.687 INFO:tasks.workunit.client.0.trial172.stdout:1.1d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585464+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:16.138950+0000 0 0 2026-02-24T18:14:28.687 INFO:tasks.workunit.client.0.trial172.stdout:1.b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586731+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.687 INFO:tasks.workunit.client.0.trial172.stdout:1.a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586791+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.687 INFO:tasks.workunit.client.0.trial172.stdout:1.9 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586985+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.687 INFO:tasks.workunit.client.0.trial172.stdout:1.8 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585828+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:22.068876+0000 0 0 2026-02-24T18:14:28.687 INFO:tasks.workunit.client.0.trial172.stdout:1.7 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.588050+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.687 INFO:tasks.workunit.client.0.trial172.stdout:1.6 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587000+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.687 INFO:tasks.workunit.client.0.trial172.stdout:1.5 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587134+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.687 INFO:tasks.workunit.client.0.trial172.stdout:1.4 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586799+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.687 INFO:tasks.workunit.client.0.trial172.stdout:1.3 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585633+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:21.108138+0000 0 0 2026-02-24T18:14:28.687 INFO:tasks.workunit.client.0.trial172.stdout:1.2 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586688+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.687 INFO:tasks.workunit.client.0.trial172.stdout:1.c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585557+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:23.105916+0000 0 0 2026-02-24T18:14:28.687 INFO:tasks.workunit.client.0.trial172.stdout:1.1 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585720+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:24.148406+0000 0 0 2026-02-24T18:14:28.688 INFO:tasks.workunit.client.0.trial172.stdout:1.1c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.583668+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:45.190446+0000 0 0 2026-02-24T18:14:28.688 INFO:tasks.workunit.client.0.trial172.stdout:1.0 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:54.908303+0000 30'100 48:150 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:44.199233+0000 0 0 2026-02-24T18:14:28.688 INFO:tasks.workunit.client.0.trial172.stdout:1.d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587174+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.688 INFO:tasks.workunit.client.0.trial172.stdout:1.e 3 0 3 0 0 2088 0 0 17 84 17 active+undersized+degraded 2026-02-24T18:13:42.587167+0000 45'101 48:150 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.688 INFO:tasks.workunit.client.0.trial172.stdout:1.f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586904+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.688 INFO:tasks.workunit.client.0.trial172.stdout:1.10 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584246+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:08.180936+0000 0 0 2026-02-24T18:14:28.688 INFO:tasks.workunit.client.0.trial172.stdout:1.11 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584341+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:05.253444+0000 0 0 2026-02-24T18:14:28.688 INFO:tasks.workunit.client.0.trial172.stdout:1.12 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584126+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:55.092490+0000 0 0 2026-02-24T18:14:28.688 INFO:tasks.workunit.client.0.trial172.stdout:1.13 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.583984+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:51.213919+0000 0 0 2026-02-24T18:14:28.688 INFO:tasks.workunit.client.0.trial172.stdout:1.14 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585448+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:19.147628+0000 0 0 2026-02-24T18:14:28.688 INFO:tasks.workunit.client.0.trial172.stdout:1.15 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585754+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.688 INFO:tasks.workunit.client.0.trial172.stdout:1.16 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585346+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:14.186892+0000 0 0 2026-02-24T18:14:28.688 INFO:tasks.workunit.client.0.trial172.stdout:1.17 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587317+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.688 INFO:tasks.workunit.client.0.trial172.stdout:1.18 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584089+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:57.124622+0000 0 0 2026-02-24T18:14:28.689 INFO:tasks.workunit.client.0.trial172.stdout:1.19 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586770+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:28.689 INFO:tasks.workunit.client.0.trial172.stdout:1.1a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584783+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:10.198312+0000 0 0 2026-02-24T18:14:28.689 INFO:tasks.workunit.client.0.trial172.stdout:1.1b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584576+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:00.121139+0000 0 0 2026-02-24T18:14:28.689 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:14:28.689 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:14:28.689 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:14:28.696 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:779: TEST_divergent_3: get_asok_path osd.1 2026-02-24T18:14:28.696 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T18:14:28.696 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T18:14:28.697 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:14:28.697 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:14:28.697 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:14:28.697 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.1.asok 2026-02-24T18:14:28.697 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:779: TEST_divergent_3: CEPH_ARGS= 2026-02-24T18:14:28.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:779: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.1.asok set_recovery_delay 100000 2026-02-24T18:14:28.764 INFO:tasks.workunit.client.0.trial172.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-24T18:14:28.770 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:781: TEST_divergent_3: ceph osd unset noup 2026-02-24T18:14:29.093 INFO:tasks.workunit.client.0.trial172.stderr:noup is unset 2026-02-24T18:14:29.107 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:783: TEST_divergent_3: wait_for_osd up 0 2026-02-24T18:14:29.107 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:14:29.107 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:14:29.107 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:14:29.107 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:14:29.107 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:14:29.107 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:14:29.107 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:14:29.108 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:14:29.108 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:14:29.337 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 37 up_thru 44 down_at 31 last_clean_interval [5,30) [v2:127.0.0.1:6800/266228372,v1:127.0.0.1:6801/266228372] [v2:127.0.0.1:6802/266228372,v1:127.0.0.1:6803/266228372] exists,up 1b2deac1-b721-42fd-ace2-60433262abf1 2026-02-24T18:14:29.339 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:14:29.339 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:14:29.339 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:14:29.339 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:784: TEST_divergent_3: wait_for_osd up 1 2026-02-24T18:14:29.339 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:14:29.339 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:14:29.339 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:14:29.339 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:14:29.340 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:14:29.340 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:14:29.340 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:14:29.340 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:14:29.341 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:14:29.561 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 50 up_thru 50 down_at 32 last_clean_interval [10,30) [v2:127.0.0.1:6818/3931471389,v1:127.0.0.1:6819/3931471389] [v2:127.0.0.1:6820/3931471389,v1:127.0.0.1:6821/3931471389] exists,up 43112d05-9157-407e-ba40-33f71a9b44d7 2026-02-24T18:14:29.562 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:14:29.562 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:14:29.563 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:14:29.563 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:785: TEST_divergent_3: wait_for_osd up 2 2026-02-24T18:14:29.563 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:14:29.563 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:14:29.563 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:14:29.563 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:14:29.563 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:14:29.563 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:14:29.564 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:14:29.564 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:14:29.564 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:14:29.772 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 39 up_thru 0 down_at 33 last_clean_interval [15,30) [v2:127.0.0.1:6810/220259212,v1:127.0.0.1:6811/220259212] [v2:127.0.0.1:6812/220259212,v1:127.0.0.1:6813/220259212] exists,up 9c16fc6e-c787-4393-91f3-0b8a744ca5a0 2026-02-24T18:14:29.772 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:14:29.772 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:14:29.773 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:14:29.773 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:787: TEST_divergent_3: ceph pg dump pgs 2026-02-24T18:14:29.974 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:14:29.974 INFO:tasks.workunit.client.0.trial172.stdout:1.1f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586737+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.975 INFO:tasks.workunit.client.0.trial172.stdout:1.1e 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585701+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.975 INFO:tasks.workunit.client.0.trial172.stdout:1.1d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585464+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:16.138950+0000 0 0 2026-02-24T18:14:29.975 INFO:tasks.workunit.client.0.trial172.stdout:1.b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586731+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.975 INFO:tasks.workunit.client.0.trial172.stdout:1.a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586791+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.975 INFO:tasks.workunit.client.0.trial172.stdout:1.9 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586985+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.975 INFO:tasks.workunit.client.0.trial172.stdout:1.8 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585828+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:22.068876+0000 0 0 2026-02-24T18:14:29.975 INFO:tasks.workunit.client.0.trial172.stdout:1.7 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.588050+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.975 INFO:tasks.workunit.client.0.trial172.stdout:1.6 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587000+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.975 INFO:tasks.workunit.client.0.trial172.stdout:1.5 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587134+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.975 INFO:tasks.workunit.client.0.trial172.stdout:1.4 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586799+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.976 INFO:tasks.workunit.client.0.trial172.stdout:1.3 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585633+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:21.108138+0000 0 0 2026-02-24T18:14:29.976 INFO:tasks.workunit.client.0.trial172.stdout:1.2 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586688+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.976 INFO:tasks.workunit.client.0.trial172.stdout:1.c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585557+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:23.105916+0000 0 0 2026-02-24T18:14:29.976 INFO:tasks.workunit.client.0.trial172.stdout:1.1 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585720+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:24.148406+0000 0 0 2026-02-24T18:14:29.976 INFO:tasks.workunit.client.0.trial172.stdout:1.1c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.583668+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:45.190446+0000 0 0 2026-02-24T18:14:29.976 INFO:tasks.workunit.client.0.trial172.stdout:1.0 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:54.908303+0000 30'100 48:150 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:44.199233+0000 0 0 2026-02-24T18:14:29.976 INFO:tasks.workunit.client.0.trial172.stdout:1.d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587174+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.976 INFO:tasks.workunit.client.0.trial172.stdout:1.e 3 0 3 0 0 2088 0 0 17 84 17 active+undersized+degraded 2026-02-24T18:13:42.587167+0000 45'101 48:150 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.976 INFO:tasks.workunit.client.0.trial172.stdout:1.f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586904+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.976 INFO:tasks.workunit.client.0.trial172.stdout:1.10 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584246+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:08.180936+0000 0 0 2026-02-24T18:14:29.976 INFO:tasks.workunit.client.0.trial172.stdout:1.11 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584341+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:05.253444+0000 0 0 2026-02-24T18:14:29.977 INFO:tasks.workunit.client.0.trial172.stdout:1.12 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584126+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:55.092490+0000 0 0 2026-02-24T18:14:29.977 INFO:tasks.workunit.client.0.trial172.stdout:1.13 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.583984+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:51.213919+0000 0 0 2026-02-24T18:14:29.977 INFO:tasks.workunit.client.0.trial172.stdout:1.14 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585448+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:19.147628+0000 0 0 2026-02-24T18:14:29.977 INFO:tasks.workunit.client.0.trial172.stdout:1.15 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585754+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.977 INFO:tasks.workunit.client.0.trial172.stdout:1.16 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585346+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:14.186892+0000 0 0 2026-02-24T18:14:29.977 INFO:tasks.workunit.client.0.trial172.stdout:1.17 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587317+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.977 INFO:tasks.workunit.client.0.trial172.stdout:1.18 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584089+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:57.124622+0000 0 0 2026-02-24T18:14:29.977 INFO:tasks.workunit.client.0.trial172.stdout:1.19 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586770+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:29.977 INFO:tasks.workunit.client.0.trial172.stdout:1.1a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584783+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:10.198312+0000 0 0 2026-02-24T18:14:29.977 INFO:tasks.workunit.client.0.trial172.stdout:1.1b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584576+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:00.121139+0000 0 0 2026-02-24T18:14:29.977 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:14:29.977 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:14:29.978 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:14:29.984 INFO:tasks.workunit.client.0.trial172.stdout:wait for peering 2026-02-24T18:14:29.984 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:788: TEST_divergent_3: echo 'wait for peering' 2026-02-24T18:14:29.984 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:789: TEST_divergent_3: ceph pg dump pgs 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:1.1f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586737+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:1.1e 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585701+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:1.1d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585464+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:16.138950+0000 0 0 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:1.b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586731+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:1.a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586791+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:1.9 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586985+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:1.8 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585828+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:22.068876+0000 0 0 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:1.7 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.588050+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:1.6 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587000+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:1.5 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587134+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:1.4 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586799+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:1.3 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585633+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:21.108138+0000 0 0 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:1.2 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586688+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:1.c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585557+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:23.105916+0000 0 0 2026-02-24T18:14:30.200 INFO:tasks.workunit.client.0.trial172.stdout:1.1 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585720+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:24.148406+0000 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.1c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.583668+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:45.190446+0000 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.0 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:54.908303+0000 30'100 48:150 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:44.199233+0000 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587174+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.e 3 0 3 0 0 2088 0 0 17 84 17 active+undersized+degraded 2026-02-24T18:13:42.587167+0000 45'101 48:150 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586904+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.10 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584246+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:08.180936+0000 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.11 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584341+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:05.253444+0000 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.12 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584126+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:55.092490+0000 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.13 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.583984+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:51.213919+0000 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.14 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585448+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:19.147628+0000 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.15 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585754+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.16 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.585346+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:14.186892+0000 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.17 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.587317+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.18 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584089+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:14:57.124622+0000 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.19 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.586770+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.1a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584783+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:10.198312+0000 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:1.1b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:13:42.584576+0000 30'100 48:149 [0,2] 0 [0,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 user requested scrub scheduled @ 2026-02-24T18:15:00.121139+0000 0 0 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:14:30.201 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:14:30.202 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:14:30.208 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:790: TEST_divergent_3: rados -p test put foo /tmp/tmp.paFLOZwKE9 2026-02-24T18:14:30.240 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:791: TEST_divergent_3: rm -f /tmp/tmp.paFLOZwKE9 2026-02-24T18:14:30.241 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:792: TEST_divergent_3: rm -f /tmp/tmp.9dNuV2JkPI 2026-02-24T18:14:30.242 INFO:tasks.workunit.client.0.trial172.stdout:killing divergent 1 2026-02-24T18:14:30.243 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:796: TEST_divergent_3: echo 'killing divergent 1' 2026-02-24T18:14:30.243 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:797: TEST_divergent_3: ceph pg dump pgs 2026-02-24T18:14:30.458 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:14:30.458 INFO:tasks.workunit.client.0.trial172.stdout:1.1f 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095718+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.459 INFO:tasks.workunit.client.0.trial172.stdout:1.1e 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095712+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.459 INFO:tasks.workunit.client.0.trial172.stdout:1.1d 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095556+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.459 INFO:tasks.workunit.client.0.trial172.stdout:1.b 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.096118+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.459 INFO:tasks.workunit.client.0.trial172.stdout:1.a 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101219+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.459 INFO:tasks.workunit.client.0.trial172.stdout:1.9 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.096231+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.459 INFO:tasks.workunit.client.0.trial172.stdout:1.8 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.096358+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.459 INFO:tasks.workunit.client.0.trial172.stdout:1.7 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101223+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.459 INFO:tasks.workunit.client.0.trial172.stdout:1.6 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101277+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.459 INFO:tasks.workunit.client.0.trial172.stdout:1.5 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101017+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.459 INFO:tasks.workunit.client.0.trial172.stdout:1.4 4 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101279+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.459 INFO:tasks.workunit.client.0.trial172.stdout:1.3 4 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101045+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.459 INFO:tasks.workunit.client.0.trial172.stdout:1.2 4 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101281+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.459 INFO:tasks.workunit.client.0.trial172.stdout:1.c 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101199+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.459 INFO:tasks.workunit.client.0.trial172.stdout:1.1 4 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.096213+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.460 INFO:tasks.workunit.client.0.trial172.stdout:1.1c 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095700+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.460 INFO:tasks.workunit.client.0.trial172.stdout:1.0 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.101125+0000 30'106 50:131 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.460 INFO:tasks.workunit.client.0.trial172.stdout:1.d 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101260+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.460 INFO:tasks.workunit.client.0.trial172.stdout:1.e 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101205+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.460 INFO:tasks.workunit.client.0.trial172.stdout:1.f 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101137+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.460 INFO:tasks.workunit.client.0.trial172.stdout:1.10 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.094750+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.460 INFO:tasks.workunit.client.0.trial172.stdout:1.11 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.094572+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.460 INFO:tasks.workunit.client.0.trial172.stdout:1.12 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.094809+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.460 INFO:tasks.workunit.client.0.trial172.stdout:1.13 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095534+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.460 INFO:tasks.workunit.client.0.trial172.stdout:1.14 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095340+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.460 INFO:tasks.workunit.client.0.trial172.stdout:1.15 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095529+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.460 INFO:tasks.workunit.client.0.trial172.stdout:1.16 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.096054+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.460 INFO:tasks.workunit.client.0.trial172.stdout:1.17 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095761+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.460 INFO:tasks.workunit.client.0.trial172.stdout:1.18 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.094700+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.461 INFO:tasks.workunit.client.0.trial172.stdout:1.19 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.094605+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.461 INFO:tasks.workunit.client.0.trial172.stdout:1.1a 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.094956+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.461 INFO:tasks.workunit.client.0.trial172.stdout:1.1b 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.094830+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:30.461 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:14:30.461 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:14:30.461 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:14:30.469 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:798: TEST_divergent_3: kill_daemons td/divergent-priors KILL osd.1 2026-02-24T18:14:30.470 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:14:30.470 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:14:30.470 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:14:30.470 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:14:30.470 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:14:30.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:14:30.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:801: TEST_divergent_3: expfile=td/divergent-priors/exp.19163.out 2026-02-24T18:14:30.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:802: TEST_divergent_3: _objectstore_tool_nodown td/divergent-priors 1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.19163.out 2026-02-24T18:14:30.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-24T18:14:30.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T18:14:30.579 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-24T18:14:30.580 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T18:14:30.580 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-24T18:14:30.580 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.19163.out 2026-02-24T18:14:30.777 INFO:tasks.workunit.client.0.trial172.stderr:Exporting 1.0 info 1.0( v 30'100 (30'90,30'100] local-lis/les=50/51 n=3 ec=22/22 lis/c=50/50 les/c/f=51/51/0 sis=50) 2026-02-24T18:14:30.778 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:023dd3d2:::existing_12:head# 2026-02-24T18:14:30.778 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:04c40a0c:::existing_93:head# 2026-02-24T18:14:30.779 INFO:tasks.workunit.client.0.trial172.stderr:Read #1:064ff5fb:::existing_36:head# 2026-02-24T18:14:30.779 INFO:tasks.workunit.client.0.trial172.stderr:Export successful 2026-02-24T18:14:30.779 INFO:tasks.workunit.client.0.trial172.stdout: marking collection for removal 2026-02-24T18:14:30.779 INFO:tasks.workunit.client.0.trial172.stdout:setting '_remove' omap key 2026-02-24T18:14:30.779 INFO:tasks.workunit.client.0.trial172.stdout:finish_remove_pgs 1.0_head removing 1.0 2026-02-24T18:14:30.780 INFO:tasks.workunit.client.0.trial172.stderr:Remove successful 2026-02-24T18:14:30.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:803: TEST_divergent_3: _objectstore_tool_nodown td/divergent-priors 1 --op import --file td/divergent-priors/exp.19163.out 2026-02-24T18:14:30.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-24T18:14:30.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T18:14:30.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-24T18:14:30.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T18:14:30.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-24T18:14:30.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op import --file td/divergent-priors/exp.19163.out 2026-02-24T18:14:31.102 INFO:tasks.workunit.client.0.trial172.stdout:get_pg_num_history pg_num_history pg_num_history(e51 pg_nums {1={22=1,44=32}} deleted_pools ) 2026-02-24T18:14:31.102 INFO:tasks.workunit.client.0.trial172.stdout:Importing pgid 1.0 2026-02-24T18:14:31.102 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:023dd3d2:::existing_12:head# 2026-02-24T18:14:31.102 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:14:31.102 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:04c40a0c:::existing_93:head# 2026-02-24T18:14:31.102 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:14:31.102 INFO:tasks.workunit.client.0.trial172.stdout:Write #1:064ff5fb:::existing_36:head# 2026-02-24T18:14:31.103 INFO:tasks.workunit.client.0.trial172.stdout:snapset 0={} 2026-02-24T18:14:31.103 INFO:tasks.workunit.client.0.trial172.stdout:write_pg epoch 51 info 1.0( v 30'100 (30'90,30'100] local-lis/les=50/51 n=3 ec=22/22 lis/c=50/50 les/c/f=51/51/0 sis=50) 2026-02-24T18:14:31.103 INFO:tasks.workunit.client.0.trial172.stdout:Import successful 2026-02-24T18:14:31.278 INFO:tasks.workunit.client.0.trial172.stdout:reviving divergent 1 2026-02-24T18:14:31.279 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:805: TEST_divergent_3: echo 'reviving divergent 1' 2026-02-24T18:14:31.279 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:806: TEST_divergent_3: ceph pg dump pgs 2026-02-24T18:14:31.482 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:14:31.482 INFO:tasks.workunit.client.0.trial172.stdout:1.1f 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095718+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.482 INFO:tasks.workunit.client.0.trial172.stdout:1.1e 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095712+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.482 INFO:tasks.workunit.client.0.trial172.stdout:1.1d 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095556+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.482 INFO:tasks.workunit.client.0.trial172.stdout:1.b 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.096118+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.482 INFO:tasks.workunit.client.0.trial172.stdout:1.a 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101219+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.482 INFO:tasks.workunit.client.0.trial172.stdout:1.9 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.096231+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.482 INFO:tasks.workunit.client.0.trial172.stdout:1.8 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.096358+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.483 INFO:tasks.workunit.client.0.trial172.stdout:1.7 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101223+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.483 INFO:tasks.workunit.client.0.trial172.stdout:1.6 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101277+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.483 INFO:tasks.workunit.client.0.trial172.stdout:1.5 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101017+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.483 INFO:tasks.workunit.client.0.trial172.stdout:1.4 4 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101279+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.483 INFO:tasks.workunit.client.0.trial172.stdout:1.3 4 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101045+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.483 INFO:tasks.workunit.client.0.trial172.stdout:1.2 4 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101281+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.483 INFO:tasks.workunit.client.0.trial172.stdout:1.c 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101199+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.483 INFO:tasks.workunit.client.0.trial172.stdout:1.1 4 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.096213+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.483 INFO:tasks.workunit.client.0.trial172.stdout:1.1c 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095700+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.483 INFO:tasks.workunit.client.0.trial172.stdout:1.0 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.101125+0000 30'106 50:131 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.483 INFO:tasks.workunit.client.0.trial172.stdout:1.d 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101260+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.483 INFO:tasks.workunit.client.0.trial172.stdout:1.e 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101205+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.484 INFO:tasks.workunit.client.0.trial172.stdout:1.f 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.101137+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.484 INFO:tasks.workunit.client.0.trial172.stdout:1.10 3 0 0 0 0 1599 0 0 16 90 16 peering 2026-02-24T18:14:29.094750+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.484 INFO:tasks.workunit.client.0.trial172.stdout:1.11 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.094572+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.484 INFO:tasks.workunit.client.0.trial172.stdout:1.12 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.094809+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.484 INFO:tasks.workunit.client.0.trial172.stdout:1.13 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095534+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.484 INFO:tasks.workunit.client.0.trial172.stdout:1.14 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095340+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.484 INFO:tasks.workunit.client.0.trial172.stdout:1.15 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095529+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.484 INFO:tasks.workunit.client.0.trial172.stdout:1.16 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.096054+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.484 INFO:tasks.workunit.client.0.trial172.stdout:1.17 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.095761+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.484 INFO:tasks.workunit.client.0.trial172.stdout:1.18 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.094700+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.484 INFO:tasks.workunit.client.0.trial172.stdout:1.19 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.094605+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.484 INFO:tasks.workunit.client.0.trial172.stdout:1.1a 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.094956+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.485 INFO:tasks.workunit.client.0.trial172.stdout:1.1b 3 0 0 0 0 1598 0 0 16 90 16 peering 2026-02-24T18:14:29.094830+0000 30'106 51:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:14:31.485 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:14:31.485 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:14:31.485 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:14:31.493 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:807: TEST_divergent_3: activate_osd td/divergent-priors 1 2026-02-24T18:14:31.493 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:14:31.493 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:14:31.493 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T18:14:31.493 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:14:31.493 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-24T18:14:31.493 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:14:31.493 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:14:31.493 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:14:31.494 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:14:31.494 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-24T18:14:31.494 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-24T18:14:31.494 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:14:31.494 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:14:31.494 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:14:31.494 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:14:31.494 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:14:31.494 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:14:31.494 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:14:31.494 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:14:31.494 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:14:31.495 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:14:31.495 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:14:31.495 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:14:31.495 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:14:31.495 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:14:31.495 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:14:31.495 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:14:31.495 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:14:31.495 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:14:31.495 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:14:31.495 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:14:31.495 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-24T18:14:31.496 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:14:31.497 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T18:14:31.497 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:14:31.497 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-24T18:14:31.497 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T18:14:31.497 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:14:31.497 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:14:31.497 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:14:31.511 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:14:31.508+0000 7effaf215a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:14:31.512 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:14:31.508+0000 7effaf215a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:14:31.513 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:14:31.508+0000 7effaf215a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:14:31.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T18:14:31.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:14:31.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:14:31.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:14:31.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:14:31.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:14:31.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:14:31.718 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:14:31.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:14:31.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:14:31.941 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 50 up_thru 50 down_at 32 last_clean_interval [10,30) [v2:127.0.0.1:6818/3931471389,v1:127.0.0.1:6819/3931471389] [v2:127.0.0.1:6820/3931471389,v1:127.0.0.1:6821/3931471389] exists,up 43112d05-9157-407e-ba40-33f71a9b44d7 2026-02-24T18:14:31.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:14:31.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:14:31.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:14:31.943 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:808: TEST_divergent_3: wait_for_osd up 1 2026-02-24T18:14:31.943 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:14:31.943 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:14:31.943 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:14:31.943 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:14:31.943 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:14:31.943 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:14:31.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:14:31.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:14:31.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:14:32.115 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:14:32.112+0000 7effaf215a40 -1 Falling back to public interface 2026-02-24T18:14:32.162 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 50 up_thru 50 down_at 32 last_clean_interval [10,30) [v2:127.0.0.1:6818/3931471389,v1:127.0.0.1:6819/3931471389] [v2:127.0.0.1:6820/3931471389,v1:127.0.0.1:6821/3931471389] exists,up 43112d05-9157-407e-ba40-33f71a9b44d7 2026-02-24T18:14:32.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:14:32.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:14:32.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:14:32.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:810: TEST_divergent_3: sleep 20 2026-02-24T18:14:32.895 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:14:32.892+0000 7effaf215a40 -1 osd.1 51 log_to_monitors true 2026-02-24T18:14:52.165 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:811: TEST_divergent_3: get_asok_path osd.1 2026-02-24T18:14:52.165 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T18:14:52.165 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T18:14:52.165 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:14:52.166 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:14:52.166 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:14:52.166 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.1.asok 2026-02-24T18:14:52.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:811: TEST_divergent_3: CEPH_ARGS= 2026-02-24T18:14:52.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:811: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.1.asok dump_ops_in_flight 2026-02-24T18:14:52.221 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:14:52.221 INFO:tasks.workunit.client.0.trial172.stdout: "ops": [], 2026-02-24T18:14:52.221 INFO:tasks.workunit.client.0.trial172.stdout: "num_ops": 0 2026-02-24T18:14:52.221 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:14:52.227 INFO:tasks.workunit.client.0.trial172.stdout:allowing recovery 2026-02-24T18:14:52.227 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:813: TEST_divergent_3: echo 'allowing recovery' 2026-02-24T18:14:52.227 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:814: TEST_divergent_3: ceph pg dump pgs 2026-02-24T18:14:52.440 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:14:52.440 INFO:tasks.workunit.client.0.trial172.stdout:1.1f 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:42.590582+0000 30'100 63:170 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.440 INFO:tasks.workunit.client.0.trial172.stdout:1.1e 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:40.590805+0000 30'100 62:171 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.440 INFO:tasks.workunit.client.0.trial172.stdout:1.1d 4 0 0 0 0 2048 0 0 16 84 16 active+clean 2026-02-24T18:14:44.455538+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-24T18:14:44.455495+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:46:41.386102+0000 4 0 2026-02-24T18:14:52.440 INFO:tasks.workunit.client.0.trial172.stdout:1.b 3 0 0 0 0 1600 0 0 10 90 10 active+clean+scrubbing 2026-02-24T18:14:47.955054+0000 30'100 63:175 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 scrubbing for 0s 0 0 2026-02-24T18:14:52.441 INFO:tasks.workunit.client.0.trial172.stdout:1.a 3 0 0 0 0 1536 0 0 16 84 16 active+clean 2026-02-24T18:14:45.291654+0000 30'100 65:187 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-24T18:14:45.291612+0000 0'0 2026-02-24T18:12:51.244607+0000 0 1 periodic scrub scheduled @ 2026-02-26T04:07:31.518470+0000 3 0 2026-02-24T18:14:52.441 INFO:tasks.workunit.client.0.trial172.stdout:1.9 5 0 0 0 0 2560 0 0 16 84 16 active+clean 2026-02-24T18:14:45.460427+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-24T18:14:45.460381+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T19:04:44.271576+0000 5 0 2026-02-24T18:14:52.441 INFO:tasks.workunit.client.0.trial172.stdout:1.8 3 0 0 0 0 1536 0 0 10 90 10 active+clean 2026-02-24T18:14:40.967545+0000 30'100 63:183 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-24T18:14:40.967490+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-26T01:22:33.236180+0000 3 0 2026-02-24T18:14:52.441 INFO:tasks.workunit.client.0.trial172.stdout:1.7 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:42.590237+0000 30'100 63:170 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.441 INFO:tasks.workunit.client.0.trial172.stdout:1.6 4 0 0 0 0 2112 0 0 17 84 17 active+clean 2026-02-24T18:14:40.591059+0000 51'101 62:162 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.441 INFO:tasks.workunit.client.0.trial172.stdout:1.5 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:38.589681+0000 30'100 66:157 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.441 INFO:tasks.workunit.client.0.trial172.stdout:1.4 4 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:33.268715+0000 30'100 63:174 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.441 INFO:tasks.workunit.client.0.trial172.stdout:1.3 4 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-24T18:14:42.590335+0000 30'100 63:180 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-24T18:14:35.874139+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.441 INFO:tasks.workunit.client.0.trial172.stdout:1.2 1 0 0 0 0 512 0 0 16 84 16 active+clean 2026-02-24T18:14:46.290580+0000 30'100 65:187 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-24T18:14:46.290528+0000 0'0 2026-02-24T18:12:51.244607+0000 0 1 periodic scrub scheduled @ 2026-02-25T23:36:08.026829+0000 1 0 2026-02-24T18:14:52.441 INFO:tasks.workunit.client.0.trial172.stdout:1.c 3 0 0 0 0 1600 0 0 14 86 14 active+clean 2026-02-24T18:14:33.268968+0000 30'100 63:175 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.441 INFO:tasks.workunit.client.0.trial172.stdout:1.1 2 0 0 0 0 1024 0 0 16 84 16 active+clean 2026-02-24T18:14:46.480668+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-24T18:14:46.480609+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-26T00:00:05.135007+0000 2 0 2026-02-24T18:14:52.442 INFO:tasks.workunit.client.0.trial172.stdout:1.1c 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:33.268779+0000 30'100 63:174 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.442 INFO:tasks.workunit.client.0.trial172.stdout:1.0 3 0 0 0 0 1536 0 0 10 90 10 active+clean 2026-02-24T18:14:42.941966+0000 30'100 63:182 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-24T18:14:42.941907+0000 0'0 2026-02-24T18:12:51.244607+0000 0 1 periodic scrub scheduled @ 2026-02-26T02:14:55.462989+0000 3 0 2026-02-24T18:14:52.442 INFO:tasks.workunit.client.0.trial172.stdout:1.d 3 0 0 0 0 1536 0 0 16 84 16 active+clean 2026-02-24T18:14:50.459035+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-24T18:14:50.458994+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T18:40:50.029836+0000 3 0 2026-02-24T18:14:52.442 INFO:tasks.workunit.client.0.trial172.stdout:1.e 3 0 0 0 0 2088 0 0 17 84 17 active+clean 2026-02-24T18:14:40.591069+0000 45'101 62:172 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.442 INFO:tasks.workunit.client.0.trial172.stdout:1.f 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:42.590640+0000 30'100 63:170 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.442 INFO:tasks.workunit.client.0.trial172.stdout:1.10 3 0 0 0 0 1536 0 0 10 90 10 active+clean 2026-02-24T18:14:46.908817+0000 30'100 63:183 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-24T18:14:46.908777+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T20:40:22.181920+0000 3 0 2026-02-24T18:14:52.442 INFO:tasks.workunit.client.0.trial172.stdout:1.11 3 0 0 0 0 1536 0 0 16 84 16 active+clean 2026-02-24T18:14:43.497321+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-24T18:14:43.497262+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T20:20:14.677297+0000 3 0 2026-02-24T18:14:52.442 INFO:tasks.workunit.client.0.trial172.stdout:1.12 1 0 0 0 0 512 0 0 16 84 16 active+clean 2026-02-24T18:14:44.321550+0000 30'100 65:187 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-24T18:14:44.321510+0000 0'0 2026-02-24T18:12:51.244607+0000 0 1 periodic scrub scheduled @ 2026-02-25T18:27:11.253709+0000 1 0 2026-02-24T18:14:52.442 INFO:tasks.workunit.client.0.trial172.stdout:1.13 3 0 0 0 0 1536 0 0 10 90 10 active+clean 2026-02-24T18:14:45.947966+0000 30'100 63:180 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-24T18:14:45.947926+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-26T04:12:34.008352+0000 3 0 2026-02-24T18:14:52.442 INFO:tasks.workunit.client.0.trial172.stdout:1.14 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-24T18:14:33.268522+0000 30'100 63:175 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.442 INFO:tasks.workunit.client.0.trial172.stdout:1.15 3 0 0 0 0 1536 0 0 16 84 16 active+clean 2026-02-24T18:14:39.511194+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-24T18:14:39.511156+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T20:38:34.832517+0000 3 0 2026-02-24T18:14:52.443 INFO:tasks.workunit.client.0.trial172.stdout:1.16 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:40.590579+0000 30'100 62:171 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.443 INFO:tasks.workunit.client.0.trial172.stdout:1.17 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:42.589930+0000 30'100 63:170 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.443 INFO:tasks.workunit.client.0.trial172.stdout:1.18 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-24T18:14:33.268871+0000 30'100 63:175 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.443 INFO:tasks.workunit.client.0.trial172.stdout:1.19 2 0 0 0 0 1024 0 0 16 84 16 active+clean 2026-02-24T18:14:40.509294+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-24T18:14:40.509256+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T20:59:38.619342+0000 2 0 2026-02-24T18:14:52.443 INFO:tasks.workunit.client.0.trial172.stdout:1.1a 4 0 0 0 0 2048 0 0 16 84 16 active+clean 2026-02-24T18:14:43.349710+0000 30'100 65:187 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-24T18:14:43.349666+0000 0'0 2026-02-24T18:12:51.244607+0000 0 1 periodic scrub scheduled @ 2026-02-25T19:07:36.435132+0000 4 0 2026-02-24T18:14:52.443 INFO:tasks.workunit.client.0.trial172.stdout:1.1b 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-24T18:14:42.590957+0000 30'100 63:173 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.443 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:14:52.443 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:14:52.443 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:14:52.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:816: TEST_divergent_3: for i in $osds 2026-02-24T18:14:52.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:818: TEST_divergent_3: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-24T18:14:52.538 INFO:tasks.workunit.client.0.trial172.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-24T18:14:52.546 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:816: TEST_divergent_3: for i in $osds 2026-02-24T18:14:52.546 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:818: TEST_divergent_3: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-24T18:14:52.629 INFO:tasks.workunit.client.0.trial172.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-24T18:14:52.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:816: TEST_divergent_3: for i in $osds 2026-02-24T18:14:52.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:818: TEST_divergent_3: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-24T18:14:52.711 INFO:tasks.workunit.client.0.trial172.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-24T18:14:52.718 INFO:tasks.workunit.client.0.trial172.stdout:reading divergent objects 2026-02-24T18:14:52.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:821: TEST_divergent_3: echo 'reading divergent objects' 2026-02-24T18:14:52.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:822: TEST_divergent_3: ceph pg dump pgs 2026-02-24T18:14:52.934 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:14:52.934 INFO:tasks.workunit.client.0.trial172.stdout:1.1f 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:42.590582+0000 30'100 63:170 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.934 INFO:tasks.workunit.client.0.trial172.stdout:1.1e 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:40.590805+0000 30'100 62:171 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.934 INFO:tasks.workunit.client.0.trial172.stdout:1.1d 4 0 0 0 0 2048 0 0 16 84 16 active+clean 2026-02-24T18:14:44.455538+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-24T18:14:44.455495+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:46:41.386102+0000 4 0 2026-02-24T18:14:52.934 INFO:tasks.workunit.client.0.trial172.stdout:1.b 3 0 0 0 0 1600 0 0 10 90 10 active+clean+scrubbing 2026-02-24T18:14:47.955054+0000 30'100 63:175 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 scrubbing for 0s 0 0 2026-02-24T18:14:52.934 INFO:tasks.workunit.client.0.trial172.stdout:1.a 3 0 0 0 0 1536 0 0 16 84 16 active+clean 2026-02-24T18:14:45.291654+0000 30'100 65:187 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-24T18:14:45.291612+0000 0'0 2026-02-24T18:12:51.244607+0000 0 1 periodic scrub scheduled @ 2026-02-26T04:07:31.518470+0000 3 0 2026-02-24T18:14:52.934 INFO:tasks.workunit.client.0.trial172.stdout:1.9 5 0 0 0 0 2560 0 0 16 84 16 active+clean 2026-02-24T18:14:45.460427+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-24T18:14:45.460381+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T19:04:44.271576+0000 5 0 2026-02-24T18:14:52.934 INFO:tasks.workunit.client.0.trial172.stdout:1.8 3 0 0 0 0 1536 0 0 10 90 10 active+clean 2026-02-24T18:14:40.967545+0000 30'100 63:183 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-24T18:14:40.967490+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-26T01:22:33.236180+0000 3 0 2026-02-24T18:14:52.935 INFO:tasks.workunit.client.0.trial172.stdout:1.7 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:42.590237+0000 30'100 63:170 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.935 INFO:tasks.workunit.client.0.trial172.stdout:1.6 4 0 0 0 0 2112 0 0 17 84 17 active+clean 2026-02-24T18:14:40.591059+0000 51'101 62:162 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.935 INFO:tasks.workunit.client.0.trial172.stdout:1.5 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:38.589681+0000 30'100 66:157 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.935 INFO:tasks.workunit.client.0.trial172.stdout:1.4 4 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:33.268715+0000 30'100 63:174 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.935 INFO:tasks.workunit.client.0.trial172.stdout:1.3 4 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-24T18:14:42.590335+0000 30'100 63:180 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-24T18:14:35.874139+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.935 INFO:tasks.workunit.client.0.trial172.stdout:1.2 1 0 0 0 0 512 0 0 16 84 16 active+clean 2026-02-24T18:14:46.290580+0000 30'100 65:187 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-24T18:14:46.290528+0000 0'0 2026-02-24T18:12:51.244607+0000 0 1 periodic scrub scheduled @ 2026-02-25T23:36:08.026829+0000 1 0 2026-02-24T18:14:52.935 INFO:tasks.workunit.client.0.trial172.stdout:1.c 3 0 0 0 0 1600 0 0 14 86 14 active+clean 2026-02-24T18:14:33.268968+0000 30'100 63:175 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.935 INFO:tasks.workunit.client.0.trial172.stdout:1.1 2 0 0 0 0 1024 0 0 16 84 16 active+clean 2026-02-24T18:14:46.480668+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-24T18:14:46.480609+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-26T00:00:05.135007+0000 2 0 2026-02-24T18:14:52.935 INFO:tasks.workunit.client.0.trial172.stdout:1.1c 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:33.268779+0000 30'100 63:174 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.935 INFO:tasks.workunit.client.0.trial172.stdout:1.0 3 0 0 0 0 1536 0 0 10 90 10 active+clean 2026-02-24T18:14:42.941966+0000 30'100 63:182 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-24T18:14:42.941907+0000 0'0 2026-02-24T18:12:51.244607+0000 0 1 periodic scrub scheduled @ 2026-02-26T02:14:55.462989+0000 3 0 2026-02-24T18:14:52.935 INFO:tasks.workunit.client.0.trial172.stdout:1.d 3 0 0 0 0 1536 0 0 16 84 16 active+clean 2026-02-24T18:14:50.459035+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-24T18:14:50.458994+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T18:40:50.029836+0000 3 0 2026-02-24T18:14:52.936 INFO:tasks.workunit.client.0.trial172.stdout:1.e 3 0 0 0 0 2088 0 0 17 84 17 active+clean 2026-02-24T18:14:40.591069+0000 45'101 62:172 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.936 INFO:tasks.workunit.client.0.trial172.stdout:1.f 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:42.590640+0000 30'100 63:170 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.936 INFO:tasks.workunit.client.0.trial172.stdout:1.10 3 0 0 0 0 1536 0 0 10 90 10 active+clean 2026-02-24T18:14:46.908817+0000 30'100 63:183 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-24T18:14:46.908777+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T20:40:22.181920+0000 3 0 2026-02-24T18:14:52.936 INFO:tasks.workunit.client.0.trial172.stdout:1.11 3 0 0 0 0 1536 0 0 16 84 16 active+clean 2026-02-24T18:14:43.497321+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-24T18:14:43.497262+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T20:20:14.677297+0000 3 0 2026-02-24T18:14:52.936 INFO:tasks.workunit.client.0.trial172.stdout:1.12 1 0 0 0 0 512 0 0 16 84 16 active+clean 2026-02-24T18:14:44.321550+0000 30'100 65:187 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-24T18:14:44.321510+0000 0'0 2026-02-24T18:12:51.244607+0000 0 1 periodic scrub scheduled @ 2026-02-25T18:27:11.253709+0000 1 0 2026-02-24T18:14:52.936 INFO:tasks.workunit.client.0.trial172.stdout:1.13 3 0 0 0 0 1536 0 0 10 90 10 active+clean 2026-02-24T18:14:45.947966+0000 30'100 63:180 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-24T18:14:45.947926+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-26T04:12:34.008352+0000 3 0 2026-02-24T18:14:52.936 INFO:tasks.workunit.client.0.trial172.stdout:1.14 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-24T18:14:33.268522+0000 30'100 63:175 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.936 INFO:tasks.workunit.client.0.trial172.stdout:1.15 3 0 0 0 0 1536 0 0 16 84 16 active+clean 2026-02-24T18:14:39.511194+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-24T18:14:39.511156+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T20:38:34.832517+0000 3 0 2026-02-24T18:14:52.936 INFO:tasks.workunit.client.0.trial172.stdout:1.16 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:40.590579+0000 30'100 62:171 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.936 INFO:tasks.workunit.client.0.trial172.stdout:1.17 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-24T18:14:42.589930+0000 30'100 63:170 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.936 INFO:tasks.workunit.client.0.trial172.stdout:1.18 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-24T18:14:33.268871+0000 30'100 63:175 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.936 INFO:tasks.workunit.client.0.trial172.stdout:1.19 2 0 0 0 0 1024 0 0 16 84 16 active+clean 2026-02-24T18:14:40.509294+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-24T18:14:40.509256+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 periodic scrub scheduled @ 2026-02-25T20:59:38.619342+0000 2 0 2026-02-24T18:14:52.937 INFO:tasks.workunit.client.0.trial172.stdout:1.1a 4 0 0 0 0 2048 0 0 16 84 16 active+clean 2026-02-24T18:14:43.349710+0000 30'100 65:187 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-24T18:14:43.349666+0000 0'0 2026-02-24T18:12:51.244607+0000 0 1 periodic scrub scheduled @ 2026-02-25T19:07:36.435132+0000 4 0 2026-02-24T18:14:52.937 INFO:tasks.workunit.client.0.trial172.stdout:1.1b 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-24T18:14:42.590957+0000 30'100 63:173 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-24T18:12:51.244607+0000 0'0 2026-02-24T18:12:51.244607+0000 0 0 queued for scrub 0 0 2026-02-24T18:14:52.937 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:14:52.937 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:14:52.937 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:14:52.943 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: expr 2 + 2 2026-02-24T18:14:52.944 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: seq 1 4 2026-02-24T18:14:52.945 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:14:52.945 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:825: TEST_divergent_3: rados -p test get existing_1 td/divergent-priors/existing 2026-02-24T18:14:52.966 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:14:52.966 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:825: TEST_divergent_3: rados -p test get existing_2 td/divergent-priors/existing 2026-02-24T18:14:52.994 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:14:52.995 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:825: TEST_divergent_3: rados -p test get existing_3 td/divergent-priors/existing 2026-02-24T18:14:53.021 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:14:53.021 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:825: TEST_divergent_3: rados -p test get existing_4 td/divergent-priors/existing 2026-02-24T18:14:53.048 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:827: TEST_divergent_3: seq 1 2 2026-02-24T18:14:53.048 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:827: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-24T18:14:53.049 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:829: TEST_divergent_3: rados -p test get newobject_1 td/divergent-priors/existing 2026-02-24T18:14:53.068 INFO:tasks.workunit.client.0.trial172.stderr:error getting test/newobject_1: (2) No such file or directory 2026-02-24T18:14:53.069 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:827: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-24T18:14:53.069 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:829: TEST_divergent_3: rados -p test get newobject_2 td/divergent-priors/existing 2026-02-24T18:14:53.097 INFO:tasks.workunit.client.0.trial172.stderr:error getting test/newobject_2: (2) No such file or directory 2026-02-24T18:14:53.099 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:831: TEST_divergent_3: rm -f td/divergent-priors/existing 2026-02-24T18:14:53.100 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:833: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2026-02-24T18:14:53.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:833: TEST_divergent_3: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-24T18:14:53.106 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f355185c640 20 _merge_object_divergent_entries: merging hoid 1:97db2a0f:::existing_2:head entries: 30'104 (30'2) modify 1:97db2a0f:::existing_2:head by client.4547.0:1 2026-02-24T18:13:06.997833+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:14:53.106 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f355185c640 20 _merge_object_divergent_entries: keeping 30'104 (30'2) modify 1:97db2a0f:::existing_2:head by client.4547.0:1 2026-02-24T18:13:06.997833+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:14:53.106 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f355185c640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-24T18:14:53.106 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f355185c640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version: 30'2 first_divergent_update: 30'104 last_divergent_update: 30'104 2026-02-24T18:14:53.106 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f355185c640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head has no more recent entries in log 2026-02-24T18:14:53.106 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f355185c640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head must be rolled back or recovered, attempting to rollback 2026-02-24T18:14:53.107 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f355185c640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head olog_can_rollback_to: 30'106 2026-02-24T18:14:53.107 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f355185c640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot rollback 30'104 (30'2) modify 1:97db2a0f:::existing_2:head by client.4547.0:1 2026-02-24T18:13:06.997833+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:14:53.107 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f355185c640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot roll back, removing and adding to missing 2026-02-24T18:14:53.107 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f355185c640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version 30'2 <= info.log_tail 30'89 2026-02-24T18:14:53.107 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354f057640 20 _merge_object_divergent_entries: merging hoid 1:70b3bf12:::existing_4:head entries: 30'106 (30'4) delete 1:70b3bf12:::existing_4:head by client.4553.0:1 2026-02-24T18:13:07.002932+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-24T18:14:53.107 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354f057640 20 _merge_object_divergent_entries: keeping 30'106 (30'4) delete 1:70b3bf12:::existing_4:head by client.4553.0:1 2026-02-24T18:13:07.002932+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-24T18:14:53.107 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354f057640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-24T18:14:53.107 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354f057640 10 _merge_object_divergent_entries: hoid 1:70b3bf12:::existing_4:head prior_version: 30'4 first_divergent_update: 30'106 last_divergent_update: 30'106 2026-02-24T18:14:53.107 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354f057640 10 _merge_object_divergent_entries: more recent entry found: 45'101 (30'4) modify 1:70b3bf12:::existing_4:head by client.4662.0:1 2026-02-24T18:13:52.689763+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false, already merged 2026-02-24T18:14:53.107 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354f057640 20 _merge_object_divergent_entries: merging hoid 1:75ac5091:::newobject_2:head entries: 30'102 (0'0) modify 1:75ac5091:::newobject_2:head by client.4541.0:1 2026-02-24T18:13:06.995284+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-24T18:14:53.107 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354f057640 20 _merge_object_divergent_entries: keeping 30'102 (0'0) modify 1:75ac5091:::newobject_2:head by client.4541.0:1 2026-02-24T18:13:06.995284+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-24T18:14:53.108 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354f057640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-24T18:14:53.108 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354f057640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version: 0'0 first_divergent_update: 30'102 last_divergent_update: 30'102 2026-02-24T18:14:53.108 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354f057640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head has no more recent entries in log 2026-02-24T18:14:53.108 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354f057640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version or op type indicates creation, deleting 2026-02-24T18:14:53.108 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354d854640 20 _merge_object_divergent_entries: merging hoid 1:8881e6d8:::existing_3:head entries: 30'105 (30'3) delete 1:8881e6d8:::existing_3:head by client.4551.0:1 2026-02-24T18:13:07.000618+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-24T18:14:53.108 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354d854640 20 _merge_object_divergent_entries: keeping 30'105 (30'3) delete 1:8881e6d8:::existing_3:head by client.4551.0:1 2026-02-24T18:13:07.000618+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-24T18:14:53.108 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354d854640 10 _merge_object_divergent_entries: hoid object_not_in_store: 1 2026-02-24T18:14:53.108 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354d854640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version: 30'3 first_divergent_update: 30'105 last_divergent_update: 30'105 2026-02-24T18:14:53.108 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354d854640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head has no more recent entries in log 2026-02-24T18:14:53.108 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354d854640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head must be rolled back or recovered, attempting to rollback 2026-02-24T18:14:53.108 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354d854640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head olog_can_rollback_to: 30'106 2026-02-24T18:14:53.108 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354d854640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot rollback 30'105 (30'3) delete 1:8881e6d8:::existing_3:head by client.4551.0:1 2026-02-24T18:13:07.000618+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-24T18:14:53.108 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354d854640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot roll back, removing and adding to missing 2026-02-24T18:14:53.109 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.408+0000 7f354d854640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version 30'3 <= info.log_tail 30'84 2026-02-24T18:14:53.109 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 20 _merge_object_divergent_entries: merging hoid 1:d858c035:::existing_1:head entries: 30'103 (30'1) modify 1:d858c035:::existing_1:head by client.4546.0:1 2026-02-24T18:13:06.997386+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:14:53.109 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 20 _merge_object_divergent_entries: keeping 30'103 (30'1) modify 1:d858c035:::existing_1:head by client.4546.0:1 2026-02-24T18:13:06.997386+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:14:53.115 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-24T18:14:53.115 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version: 30'1 first_divergent_update: 30'103 last_divergent_update: 30'103 2026-02-24T18:14:53.115 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head has no more recent entries in log 2026-02-24T18:14:53.115 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head must be rolled back or recovered, attempting to rollback 2026-02-24T18:14:53.115 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head olog_can_rollback_to: 30'106 2026-02-24T18:14:53.115 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot rollback 30'103 (30'1) modify 1:d858c035:::existing_1:head by client.4546.0:1 2026-02-24T18:13:06.997386+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:14:53.115 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot roll back, removing and adding to missing 2026-02-24T18:14:53.115 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version 30'1 <= info.log_tail 30'90 2026-02-24T18:14:53.115 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 20 _merge_object_divergent_entries: merging hoid 1:da59f6e7:::newobject_1:head entries: 30'101 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4540.0:1 2026-02-24T18:13:06.994908+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-24T18:14:53.115 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 20 _merge_object_divergent_entries: keeping 30'101 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4540.0:1 2026-02-24T18:13:06.994908+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-24T18:14:53.116 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-24T18:14:53.116 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version: 0'0 first_divergent_update: 30'101 last_divergent_update: 30'101 2026-02-24T18:14:53.116 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head has no more recent entries in log 2026-02-24T18:14:53.116 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:14:29.412+0000 7f354c852640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version or op type indicates creation, deleting 2026-02-24T18:14:53.116 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:835: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2026-02-24T18:14:53.117 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:835: TEST_divergent_3: grep -q '_merge_object_divergent_entries: more recent entry found:' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-24T18:14:53.120 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:841: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2026-02-24T18:14:53.121 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:841: TEST_divergent_3: grep -q '_merge_object_divergent_entries.*prior_version or op type indicates creation' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-24T18:14:53.124 INFO:tasks.workunit.client.0.trial172.stdout:success 2026-02-24T18:14:53.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:846: TEST_divergent_3: echo success 2026-02-24T18:14:53.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:848: TEST_divergent_3: rm td/divergent-priors/td/divergent-priors/exp.19163.out 2026-02-24T18:14:53.125 INFO:tasks.workunit.client.0.trial172.stderr:rm: cannot remove 'td/divergent-priors/td/divergent-priors/exp.19163.out': No such file or directory 2026-02-24T18:14:53.125 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:849: TEST_divergent_3: delete_pool test 2026-02-24T18:14:53.125 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=test 2026-02-24T18:14:53.125 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-24T18:14:53.379 INFO:tasks.workunit.client.0.trial172.stderr:pool 'test' does not exist 2026-02-24T18:14:53.390 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:850: TEST_divergent_3: kill_daemons td/divergent-priors 2026-02-24T18:14:53.391 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:14:53.391 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:14:53.392 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:14:53.392 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:14:53.392 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:14:58.708 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:14:58.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-24T18:14:58.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-24T18:14:58.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:14:58.709 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-24T18:14:58.710 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:14:58.710 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:14:58.710 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:14:58.710 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:14:58.710 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:14:58.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:14:58.717 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:14:58.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:14:58.718 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:14:58.720 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:14:58.720 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:14:58.720 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:14:58.721 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:14:58.722 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:14:58.722 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:14:58.722 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:14:58.723 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:14:58.724 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:14:58.724 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-24T18:14:58.752 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:14:58.752 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:14:58.752 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:14:58.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19163 2026-02-24T18:14:58.754 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:14:58.754 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:14:58.754 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-24T18:14:58.754 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-24T18:14:58.755 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/divergent-priors 2026-02-24T18:14:58.755 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/divergent-priors 2026-02-24T18:14:58.755 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-24T18:14:58.755 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:14:58.755 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-24T18:14:58.755 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:14:58.755 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:14:58.755 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:14:58.755 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:14:58.755 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:14:58.756 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:14:58.757 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:14:58.757 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:14:58.758 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:14:58.759 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:14:58.759 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:14:58.759 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:14:58.760 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:14:58.760 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:14:58.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:14:58.761 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:14:58.762 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:14:58.763 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:14:58.763 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-24T18:14:58.765 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:14:58.765 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:14:58.765 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:14:58.765 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19163 2026-02-24T18:14:58.766 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:14:58.767 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:14:58.767 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/divergent-priors 2026-02-24T18:14:58.768 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T18:14:58.768 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:14:58.768 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:14:58.768 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19163 2026-02-24T18:14:58.770 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T18:14:58.771 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T18:14:58.771 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T18:14:58.771 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-24T18:14:58.771 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_ec td/divergent-priors 2026-02-24T18:14:58.771 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:227: TEST_divergent_ec: local dir=td/divergent-priors 2026-02-24T18:14:58.772 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:229: TEST_divergent_ec: file_with_random_data 2026-02-24T18:14:58.772 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: file_with_random_data: local size_bytes=512 2026-02-24T18:14:58.772 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: file_with_random_data: mktemp 2026-02-24T18:14:58.773 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: file_with_random_data: local file=/tmp/tmp.oW6cFhP1GX 2026-02-24T18:14:58.773 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2374: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.oW6cFhP1GX bs=512 count=1 2026-02-24T18:14:58.774 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records in 2026-02-24T18:14:58.774 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records out 2026-02-24T18:14:58.774 INFO:tasks.workunit.client.0.trial172.stderr:512 bytes copied, 0.000112856 s, 4.5 MB/s 2026-02-24T18:14:58.774 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: file_with_random_data: printf %s /tmp/tmp.oW6cFhP1GX 2026-02-24T18:14:58.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:229: TEST_divergent_ec: local dummyfile=/tmp/tmp.oW6cFhP1GX 2026-02-24T18:14:58.775 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:231: TEST_divergent_ec: file_with_random_data 1000 2026-02-24T18:14:58.775 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: file_with_random_data: local size_bytes=1000 2026-02-24T18:14:58.775 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: file_with_random_data: mktemp 2026-02-24T18:14:58.776 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: file_with_random_data: local file=/tmp/tmp.ltmCGn22fu 2026-02-24T18:14:58.776 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2374: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.ltmCGn22fu bs=1000 count=1 2026-02-24T18:14:58.777 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records in 2026-02-24T18:14:58.777 INFO:tasks.workunit.client.0.trial172.stderr:1+0 records out 2026-02-24T18:14:58.778 INFO:tasks.workunit.client.0.trial172.stderr:1000 bytes (1.0 kB) copied, 8.6048e-05 s, 11.6 MB/s 2026-02-24T18:14:58.778 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: file_with_random_data: printf %s /tmp/tmp.ltmCGn22fu 2026-02-24T18:14:58.778 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:231: TEST_divergent_ec: local dummyfile2=/tmp/tmp.ltmCGn22fu 2026-02-24T18:14:58.778 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:233: TEST_divergent_ec: local num_osds=3 2026-02-24T18:14:58.779 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:234: TEST_divergent_ec: expr 3 - 1 2026-02-24T18:14:58.780 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:234: TEST_divergent_ec: seq 0 2 2026-02-24T18:14:58.781 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:234: TEST_divergent_ec: local 'osds=0 2026-02-24T18:14:58.781 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:14:58.781 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:14:58.781 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:235: TEST_divergent_ec: run_mon td/divergent-priors a 2026-02-24T18:14:58.781 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/divergent-priors 2026-02-24T18:14:58.782 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T18:14:58.782 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T18:14:58.782 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T18:14:58.782 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/divergent-priors/a 2026-02-24T18:14:58.782 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-24T18:14:58.807 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T18:14:58.807 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:14:58.807 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:14:58.807 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:14:58.807 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:14:58.807 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:14:58.807 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:14:58.808 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T18:14:58.837 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T18:14:58.838 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T18:14:58.839 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:14:58.839 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:14:58.839 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T18:14:58.839 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T18:14:58.839 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:14:58.839 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:14:58.839 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:14:58.840 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:14:58.840 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:14:58.841 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:14:58.841 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-mon.a.asok 2026-02-24T18:14:58.841 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:14:58.841 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19163/ceph-mon.a.asok config get fsid 2026-02-24T18:14:58.894 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T18:14:58.895 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:14:58.895 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:14:58.895 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T18:14:58.895 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T18:14:58.896 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:14:58.896 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:14:58.896 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:14:58.897 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:14:58.897 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:14:58.897 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:14:58.897 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-mon.a.asok 2026-02-24T18:14:58.897 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:14:58.897 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19163/ceph-mon.a.asok config get mon_host 2026-02-24T18:14:58.950 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:236: TEST_divergent_ec: run_mgr td/divergent-priors x 2026-02-24T18:14:58.950 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/divergent-priors 2026-02-24T18:14:58.950 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T18:14:58.950 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T18:14:58.950 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T18:14:58.950 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/divergent-priors/x 2026-02-24T18:14:58.950 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T18:14:59.063 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T18:14:59.063 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:14:59.063 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:14:59.063 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:14:59.063 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:14:59.063 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:14:59.064 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:14:59.064 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:14:59.065 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:14:59.087 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:237: TEST_divergent_ec: for i in $osds 2026-02-24T18:14:59.088 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:239: TEST_divergent_ec: run_osd td/divergent-priors 0 2026-02-24T18:14:59.088 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-24T18:14:59.088 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:14:59.088 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T18:14:59.088 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:14:59.088 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/0 2026-02-24T18:14:59.088 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:14:59.088 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:14:59.089 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:14:59.089 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:14:59.089 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-24T18:14:59.089 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-24T18:14:59.089 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:14:59.089 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:14:59.089 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:14:59.089 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:14:59.089 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:14:59.089 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:14:59.089 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:14:59.090 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:14:59.090 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:14:59.090 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:14:59.090 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:14:59.090 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:14:59.090 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:14:59.090 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:14:59.090 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:14:59.090 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:14:59.090 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:14:59.090 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:14:59.090 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:14:59.091 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:14:59.091 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:14:59.091 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:14:59.091 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/0 2026-02-24T18:14:59.091 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:14:59.091 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=93b13c7c-e44d-45ef-876f-6c2e68859b9a 2026-02-24T18:14:59.092 INFO:tasks.workunit.client.0.trial172.stdout:add osd0 93b13c7c-e44d-45ef-876f-6c2e68859b9a 2026-02-24T18:14:59.092 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 93b13c7c-e44d-45ef-876f-6c2e68859b9a' 2026-02-24T18:14:59.092 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:14:59.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCj6p1pIH0ABhAAK/CNpd2p7hc9R8b7qfz/ew== 2026-02-24T18:14:59.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCj6p1pIH0ABhAAK/CNpd2p7hc9R8b7qfz/ew=="}' 2026-02-24T18:14:59.103 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 93b13c7c-e44d-45ef-876f-6c2e68859b9a -i td/divergent-priors/0/new.json 2026-02-24T18:14:59.218 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:14:59.226 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/0/new.json 2026-02-24T18:14:59.227 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCj6p1pIH0ABhAAK/CNpd2p7hc9R8b7qfz/ew== --osd-uuid 93b13c7c-e44d-45ef-876f-6c2e68859b9a 2026-02-24T18:14:59.247 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:14:59.243+0000 7f7f4dd14a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:14:59.248 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:14:59.243+0000 7f7f4dd14a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:14:59.249 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:14:59.243+0000 7f7f4dd14a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:14:59.250 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:14:59.243+0000 7f7f4dd14a40 -1 bdev(0x55d806640400 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-24T18:14:59.250 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:14:59.243+0000 7f7f4dd14a40 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-24T18:15:00.002 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-24T18:15:00.002 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:15:00.003 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T18:15:00.003 INFO:tasks.workunit.client.0.trial172.stdout:adding osd0 key to auth repository 2026-02-24T18:15:00.004 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:15:00.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T18:15:00.128 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:15:00.129 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:15:00.129 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:15:00.129 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:15:00.129 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:15:00.159 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:00.155+0000 7f8668660a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:00.161 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:00.155+0000 7f8668660a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:00.162 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:00.155+0000 7f8668660a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:00.244 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T18:15:00.245 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:15:00.245 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:15:00.245 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:15:00.245 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:15:00.245 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:15:00.245 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:15:00.246 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:15:00.246 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:15:00.246 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:15:00.357 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:15:00.588 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:00.583+0000 7f8668660a40 -1 Falling back to public interface 2026-02-24T18:15:01.303 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:01.299+0000 7f8668660a40 -1 osd.0 0 log_to_monitors true 2026-02-24T18:15:01.358 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:15:01.359 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:15:01.359 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:15:01.359 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:15:01.360 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:15:01.360 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:15:01.471 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:15:02.473 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:15:02.473 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:15:02.473 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:15:02.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:15:02.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:15:02.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:15:02.683 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3069539794,v1:127.0.0.1:6801/3069539794] [v2:127.0.0.1:6802/3069539794,v1:127.0.0.1:6803/3069539794] exists,up 93b13c7c-e44d-45ef-876f-6c2e68859b9a 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:237: TEST_divergent_ec: for i in $osds 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:239: TEST_divergent_ec: run_osd td/divergent-priors 1 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/1 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:15:02.684 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:15:02.685 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:15:02.685 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:15:02.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:15:02.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:15:02.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:15:02.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:15:02.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:15:02.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:15:02.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:15:02.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:15:02.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:15:02.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:15:02.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:15:02.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:15:02.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/1 2026-02-24T18:15:02.687 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:15:02.688 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a1643499-a87c-4291-842c-c57767e846a8 2026-02-24T18:15:02.688 INFO:tasks.workunit.client.0.trial172.stdout:add osd1 a1643499-a87c-4291-842c-c57767e846a8 2026-02-24T18:15:02.688 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 a1643499-a87c-4291-842c-c57767e846a8' 2026-02-24T18:15:02.688 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:15:02.703 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCm6p1pVcnMKRAAcLlA8XdiG5NiMqTpDmMAKA== 2026-02-24T18:15:02.703 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCm6p1pVcnMKRAAcLlA8XdiG5NiMqTpDmMAKA=="}' 2026-02-24T18:15:02.703 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a1643499-a87c-4291-842c-c57767e846a8 -i td/divergent-priors/1/new.json 2026-02-24T18:15:02.916 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:15:02.925 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/1/new.json 2026-02-24T18:15:02.925 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCm6p1pVcnMKRAAcLlA8XdiG5NiMqTpDmMAKA== --osd-uuid a1643499-a87c-4291-842c-c57767e846a8 2026-02-24T18:15:02.946 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:02.939+0000 7fda90a65a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:02.947 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:02.943+0000 7fda90a65a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:02.948 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:02.943+0000 7fda90a65a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:02.949 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:02.943+0000 7fda90a65a40 -1 bdev(0x56503799a400 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-24T18:15:02.949 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:02.943+0000 7fda90a65a40 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-24T18:15:03.613 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-24T18:15:03.613 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:15:03.613 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T18:15:03.614 INFO:tasks.workunit.client.0.trial172.stdout:adding osd1 key to auth repository 2026-02-24T18:15:03.614 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:15:03.913 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:15:03.914 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T18:15:03.914 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:15:03.914 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:15:03.914 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:15:03.915 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:15:03.944 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:03.939+0000 7f7787ad9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:03.946 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:03.939+0000 7f7787ad9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:03.947 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:03.943+0000 7f7787ad9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:04.133 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T18:15:04.133 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:15:04.133 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:15:04.133 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:15:04.133 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:15:04.133 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:15:04.134 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:15:04.134 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:15:04.134 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:15:04.134 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:15:04.345 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:15:04.388 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:04.383+0000 7f7787ad9a40 -1 Falling back to public interface 2026-02-24T18:15:05.043 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:05.039+0000 7f7787ad9a40 -1 osd.1 0 log_to_monitors true 2026-02-24T18:15:05.346 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:15:05.347 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:15:05.347 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:15:05.348 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:15:05.348 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:15:05.348 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:15:05.570 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:15:06.571 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:15:06.572 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:15:06.572 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:15:06.573 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:15:06.573 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:15:06.573 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:15:06.783 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/427574815,v1:127.0.0.1:6811/427574815] [v2:127.0.0.1:6812/427574815,v1:127.0.0.1:6813/427574815] exists,up a1643499-a87c-4291-842c-c57767e846a8 2026-02-24T18:15:06.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:15:06.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:15:06.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:15:06.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:237: TEST_divergent_ec: for i in $osds 2026-02-24T18:15:06.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:239: TEST_divergent_ec: run_osd td/divergent-priors 2 2026-02-24T18:15:06.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-24T18:15:06.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:15:06.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T18:15:06.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:15:06.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/2 2026-02-24T18:15:06.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:15:06.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:15:06.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:15:06.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:15:06.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-24T18:15:06.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-24T18:15:06.789 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:15:06.789 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:15:06.789 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:15:06.789 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:15:06.789 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:15:06.789 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:15:06.789 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:15:06.789 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:15:06.789 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:15:06.789 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:15:06.789 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:15:06.789 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:15:06.789 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:15:06.790 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:15:06.790 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:15:06.790 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:15:06.790 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:15:06.790 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:15:06.790 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:15:06.790 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:15:06.790 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:15:06.790 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:15:06.790 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/2 2026-02-24T18:15:06.790 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:15:06.791 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4e270c3f-1799-484f-8fb2-c5e0604b5c26 2026-02-24T18:15:06.791 INFO:tasks.workunit.client.0.trial172.stdout:add osd2 4e270c3f-1799-484f-8fb2-c5e0604b5c26 2026-02-24T18:15:06.792 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 4e270c3f-1799-484f-8fb2-c5e0604b5c26' 2026-02-24T18:15:06.792 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:15:06.803 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCq6p1p3XjGLxAA1OdQtbyQ06gcA/fDvEgsdg== 2026-02-24T18:15:06.804 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCq6p1p3XjGLxAA1OdQtbyQ06gcA/fDvEgsdg=="}' 2026-02-24T18:15:06.804 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4e270c3f-1799-484f-8fb2-c5e0604b5c26 -i td/divergent-priors/2/new.json 2026-02-24T18:15:07.027 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:15:07.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/2/new.json 2026-02-24T18:15:07.036 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCq6p1p3XjGLxAA1OdQtbyQ06gcA/fDvEgsdg== --osd-uuid 4e270c3f-1799-484f-8fb2-c5e0604b5c26 2026-02-24T18:15:07.055 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:07.051+0000 7f1e6187aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:07.057 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:07.051+0000 7f1e6187aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:07.058 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:07.051+0000 7f1e6187aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:07.058 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:07.055+0000 7f1e6187aa40 -1 bdev(0x559cac12a400 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-24T18:15:07.058 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:07.055+0000 7f1e6187aa40 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-24T18:15:07.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-24T18:15:07.698 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:15:07.699 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T18:15:07.699 INFO:tasks.workunit.client.0.trial172.stdout:adding osd2 key to auth repository 2026-02-24T18:15:07.700 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:15:07.989 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:15:07.990 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T18:15:07.990 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:15:07.991 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:15:07.991 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:15:07.991 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:15:08.018 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:08.015+0000 7fb943b69a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:08.020 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:08.015+0000 7fb943b69a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:08.021 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:08.015+0000 7fb943b69a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:08.213 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T18:15:08.213 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:15:08.214 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:15:08.214 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:15:08.214 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:15:08.214 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:15:08.214 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:15:08.215 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:15:08.215 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:15:08.215 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:15:08.419 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:08.415+0000 7fb943b69a40 -1 Falling back to public interface 2026-02-24T18:15:08.437 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:15:09.084 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:09.079+0000 7fb943b69a40 -1 osd.2 0 log_to_monitors true 2026-02-24T18:15:09.440 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:15:09.440 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:15:09.440 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:15:09.441 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:15:09.442 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:15:09.442 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:15:09.667 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:15:10.668 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:15:10.669 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:15:10.669 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:15:10.670 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:15:10.670 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:15:10.670 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:15:10.884 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1505594968,v1:127.0.0.1:6819/1505594968] [v2:127.0.0.1:6820/1505594968,v1:127.0.0.1:6821/1505594968] exists,up 4e270c3f-1799-484f-8fb2-c5e0604b5c26 2026-02-24T18:15:10.885 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:15:10.885 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:15:10.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:15:10.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:242: TEST_divergent_ec: ceph osd set noout 2026-02-24T18:15:11.174 INFO:tasks.workunit.client.0.trial172.stderr:noout is set 2026-02-24T18:15:11.186 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:243: TEST_divergent_ec: ceph osd set noin 2026-02-24T18:15:11.477 INFO:tasks.workunit.client.0.trial172.stderr:noin is set 2026-02-24T18:15:11.487 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:244: TEST_divergent_ec: ceph osd set nodown 2026-02-24T18:15:11.783 INFO:tasks.workunit.client.0.trial172.stderr:nodown is set 2026-02-24T18:15:11.797 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:245: TEST_divergent_ec: create_ec_pool test true k=2 m=1 2026-02-24T18:15:11.797 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2550: create_ec_pool: local pool_name=test 2026-02-24T18:15:11.797 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2551: create_ec_pool: shift 2026-02-24T18:15:11.797 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2552: create_ec_pool: local allow_overwrites=true 2026-02-24T18:15:11.797 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2553: create_ec_pool: shift 2026-02-24T18:15:11.797 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2555: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-24T18:15:12.049 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2557: create_ec_pool: create_pool test 1 1 erasure myprofile 2026-02-24T18:15:12.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 erasure myprofile 2026-02-24T18:15:12.444 INFO:tasks.workunit.client.0.trial172.stderr:pool 'test' already exists 2026-02-24T18:15:12.453 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T18:15:13.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2559: create_ec_pool: '[' true = true ']' 2026-02-24T18:15:13.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2560: create_ec_pool: ceph osd pool set test allow_ec_overwrites true 2026-02-24T18:15:13.761 INFO:tasks.workunit.client.0.trial172.stderr:set pool 1 allow_ec_overwrites to true 2026-02-24T18:15:13.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2563: create_ec_pool: wait_for_clean 2026-02-24T18:15:13.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:15:13.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:15:13.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:15:13.774 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T18:15:13.775 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:15:13.775 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:15:13.775 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:15:13.775 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:15:13.776 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:15:13.891 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T18:15:13.891 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:15:13.891 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:15:13.891 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:15:13.891 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:15:13.891 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:15:14.119 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:15:14.120 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:15:14.120 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:15:14.120 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:15:14.120 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:15:14.120 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:15:14.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836485 2026-02-24T18:15:14.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836485 2026-02-24T18:15:14.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836485' 2026-02-24T18:15:14.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:15:14.203 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:15:14.286 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672964 2026-02-24T18:15:14.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672964 2026-02-24T18:15:14.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-24T18:15:14.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:15:14.287 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:15:14.371 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509443 2026-02-24T18:15:14.371 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509443 2026-02-24T18:15:14.371 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-64424509443' 2026-02-24T18:15:14.371 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:15:14.372 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836485 2026-02-24T18:15:14.372 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:15:14.372 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:15:14.373 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836485 2026-02-24T18:15:14.373 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:15:14.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836485 2026-02-24T18:15:14.375 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 21474836485 2026-02-24T18:15:14.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-24T18:15:14.376 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:15:14.595 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836483 -lt 21474836485 2026-02-24T18:15:14.595 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:15:15.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:15:15.597 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:15:15.815 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-24T18:15:15.815 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:15:15.815 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672964 2026-02-24T18:15:15.816 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:15:15.817 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:15:15.817 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672964 2026-02-24T18:15:15.817 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:15:15.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672964 2026-02-24T18:15:15.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-24T18:15:15.819 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 42949672964 2026-02-24T18:15:15.820 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:15:16.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-24T18:15:16.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:15:16.040 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509443 2026-02-24T18:15:16.040 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:15:16.041 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:15:16.042 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509443 2026-02-24T18:15:16.042 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:15:16.043 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509443 2026-02-24T18:15:16.043 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 64424509443 2026-02-24T18:15:16.044 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509443' 2026-02-24T18:15:16.044 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:15:16.253 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509444 -lt 64424509443 2026-02-24T18:15:16.253 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:15:16.254 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:15:16.254 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:15:16.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:15:16.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:15:16.542 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:15:16.542 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:15:16.542 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:15:16.542 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:15:16.542 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:15:16.542 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:15:16.755 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-24T18:15:16.756 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:15:16.757 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:15:16.757 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:15:17.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-24T18:15:17.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T18:15:17.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T18:15:17.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2564: create_ec_pool: return 0 2026-02-24T18:15:17.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:247: TEST_divergent_ec: flush_pg_stats 2026-02-24T18:15:17.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:15:17.051 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:15:17.264 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:15:17.265 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:15:17.265 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:15:17.265 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:15:17.265 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:15:17.265 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:15:17.349 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836487 2026-02-24T18:15:17.349 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836487 2026-02-24T18:15:17.349 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836487' 2026-02-24T18:15:17.349 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:15:17.349 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:15:17.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672967 2026-02-24T18:15:17.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672967 2026-02-24T18:15:17.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836487 1-42949672967' 2026-02-24T18:15:17.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:15:17.432 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:15:17.516 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509446 2026-02-24T18:15:17.516 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509446 2026-02-24T18:15:17.516 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836487 1-42949672967 2-64424509446' 2026-02-24T18:15:17.517 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:15:17.517 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836487 2026-02-24T18:15:17.517 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:15:17.517 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:15:17.518 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836487 2026-02-24T18:15:17.518 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:15:17.519 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836487 2026-02-24T18:15:17.520 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 21474836487 2026-02-24T18:15:17.520 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-24T18:15:17.520 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:15:17.743 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-24T18:15:17.744 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:15:18.744 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:15:18.745 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:15:18.961 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-24T18:15:18.961 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:15:19.962 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T18:15:19.963 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:15:20.180 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836488 -lt 21474836487 2026-02-24T18:15:20.181 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:15:20.181 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672967 2026-02-24T18:15:20.181 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:15:20.182 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:15:20.183 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672967 2026-02-24T18:15:20.183 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:15:20.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672967 2026-02-24T18:15:20.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672967' 2026-02-24T18:15:20.185 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 42949672967 2026-02-24T18:15:20.185 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:15:20.418 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672967 -lt 42949672967 2026-02-24T18:15:20.418 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:15:20.419 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509446 2026-02-24T18:15:20.419 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:15:20.420 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:15:20.421 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509446 2026-02-24T18:15:20.421 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:15:20.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509446 2026-02-24T18:15:20.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509446' 2026-02-24T18:15:20.423 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 64424509446 2026-02-24T18:15:20.423 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:15:20.651 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509446 -lt 64424509446 2026-02-24T18:15:20.652 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:248: TEST_divergent_ec: wait_for_clean 2026-02-24T18:15:20.652 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:15:20.652 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:15:20.652 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:15:20.652 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T18:15:20.652 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:15:20.652 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:15:20.653 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:15:20.653 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:15:20.653 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:15:20.758 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T18:15:20.758 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:15:20.758 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:15:20.758 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:15:20.758 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:15:20.758 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:15:20.987 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:15:20.987 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:15:20.988 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:15:20.988 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:15:20.988 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:15:20.988 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:15:21.072 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836490 2026-02-24T18:15:21.072 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836490 2026-02-24T18:15:21.072 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490' 2026-02-24T18:15:21.072 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:15:21.073 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:15:21.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672969 2026-02-24T18:15:21.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672969 2026-02-24T18:15:21.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-42949672969' 2026-02-24T18:15:21.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:15:21.152 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:15:21.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509449 2026-02-24T18:15:21.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509449 2026-02-24T18:15:21.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-42949672969 2-64424509449' 2026-02-24T18:15:21.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:15:21.236 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836490 2026-02-24T18:15:21.236 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:15:21.237 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:15:21.238 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836490 2026-02-24T18:15:21.238 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:15:21.240 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836490 2026-02-24T18:15:21.240 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-24T18:15:21.240 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 21474836490 2026-02-24T18:15:21.240 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:15:21.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836490 -lt 21474836490 2026-02-24T18:15:21.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:15:21.465 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672969 2026-02-24T18:15:21.465 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:15:21.466 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:15:21.466 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672969 2026-02-24T18:15:21.466 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:15:21.468 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672969 2026-02-24T18:15:21.468 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672969' 2026-02-24T18:15:21.468 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 42949672969 2026-02-24T18:15:21.469 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:15:21.688 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672970 -lt 42949672969 2026-02-24T18:15:21.688 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:15:21.689 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509449 2026-02-24T18:15:21.689 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:15:21.690 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:15:21.690 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509449 2026-02-24T18:15:21.690 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:15:21.692 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509449 2026-02-24T18:15:21.692 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509449' 2026-02-24T18:15:21.692 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 64424509449 2026-02-24T18:15:21.693 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:15:21.922 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509449 -lt 64424509449 2026-02-24T18:15:21.922 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:15:21.923 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:15:21.923 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:15:22.219 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:15:22.219 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:15:22.220 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:15:22.220 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:15:22.220 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:15:22.220 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:15:22.220 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:15:22.220 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:15:22.437 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-24T18:15:22.438 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:15:22.438 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:15:22.439 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:15:22.730 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-24T18:15:22.731 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T18:15:22.731 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T18:15:22.731 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:251: TEST_divergent_ec: ceph pg dump pgs --format=json 2026-02-24T18:15:22.732 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:251: TEST_divergent_ec: jq '.pg_stats[0].up_primary' 2026-02-24T18:15:22.944 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:15:22.954 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:251: TEST_divergent_ec: local divergent=1 2026-02-24T18:15:22.954 INFO:tasks.workunit.client.0.trial172.stdout:primary and soon to be divergent is 1 2026-02-24T18:15:22.955 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:252: TEST_divergent_ec: echo 'primary and soon to be divergent is 1' 2026-02-24T18:15:22.955 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:253: TEST_divergent_ec: ceph pg dump pgs 2026-02-24T18:15:23.157 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:15:23.157 INFO:tasks.workunit.client.0.trial172.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T18:15:13.145460+0000 0'0 29:20 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:15:12.387431+0000 0'0 2026-02-24T18:15:12.387431+0000 0 0 periodic scrub scheduled @ 2026-02-25T22:44:02.327529+0000 0 0 2026-02-24T18:15:23.157 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:15:23.157 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:15:23.157 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:15:23.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:254: TEST_divergent_ec: local non_divergent= 2026-02-24T18:15:23.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:255: TEST_divergent_ec: for i in $osds 2026-02-24T18:15:23.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:257: TEST_divergent_ec: '[' 0 = 1 ']' 2026-02-24T18:15:23.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:260: TEST_divergent_ec: non_divergent=' 0' 2026-02-24T18:15:23.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:255: TEST_divergent_ec: for i in $osds 2026-02-24T18:15:23.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:257: TEST_divergent_ec: '[' 1 = 1 ']' 2026-02-24T18:15:23.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:258: TEST_divergent_ec: continue 2026-02-24T18:15:23.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:255: TEST_divergent_ec: for i in $osds 2026-02-24T18:15:23.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:257: TEST_divergent_ec: '[' 2 = 1 ']' 2026-02-24T18:15:23.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:260: TEST_divergent_ec: non_divergent=' 0 2' 2026-02-24T18:15:23.167 INFO:tasks.workunit.client.0.trial172.stdout:writing initial objects 2026-02-24T18:15:23.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:263: TEST_divergent_ec: echo 'writing initial objects' 2026-02-24T18:15:23.167 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: seq 1 100 2026-02-24T18:15:23.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_1 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.192 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.192 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_2 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.221 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.221 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_3 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.250 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.250 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_4 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.280 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.280 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_5 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.308 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.309 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_6 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.336 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.336 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_7 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.365 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.365 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_8 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.393 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.393 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_9 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_10 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.451 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_11 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.480 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.480 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_12 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.508 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.508 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_13 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.537 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.537 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_14 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.565 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.565 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_15 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.593 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.593 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_16 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.620 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.620 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_17 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.649 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.649 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_18 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.676 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.676 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_19 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.705 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.705 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_20 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.733 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.733 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_21 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_22 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.789 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.789 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_23 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.817 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.818 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_24 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.842 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.842 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_25 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.871 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.871 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_26 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_27 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.929 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.929 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_28 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.957 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.957 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_29 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:23.986 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:23.986 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_30 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.015 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.015 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_31 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.044 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.044 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_32 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.072 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.072 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_33 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_34 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.129 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_35 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.157 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.157 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_36 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.184 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_37 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.212 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.212 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_38 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.240 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.240 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_39 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.268 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.268 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_40 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.296 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.297 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_41 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_42 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.344 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.344 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_43 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_44 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.401 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.401 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_45 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.429 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.429 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_46 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.458 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.458 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_47 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_48 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.513 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.514 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_49 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_50 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.563 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.563 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_51 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_52 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.620 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.621 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_53 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.649 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.649 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_54 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.677 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.677 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_55 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.706 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.706 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_56 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.733 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.733 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_57 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_58 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.789 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.789 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_59 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.817 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.817 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_60 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.845 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_61 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.866 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.866 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_62 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.896 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.896 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_63 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.925 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.925 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_64 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.955 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.955 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_65 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:24.982 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:24.982 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_66 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.011 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.012 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_67 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.041 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.041 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_68 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.069 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.069 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_69 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.097 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.097 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_70 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.125 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.125 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_71 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_72 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.181 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.181 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_73 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.210 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.211 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_74 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.230 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.230 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_75 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_76 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.288 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.288 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_77 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.315 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.315 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_78 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.344 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.344 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_79 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_80 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.402 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.402 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_81 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_82 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.461 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.461 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_83 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.489 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.489 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_84 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.510 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.510 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_85 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.531 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_86 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.552 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.553 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_87 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.574 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.574 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_88 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_89 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.630 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.630 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_90 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.659 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.659 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_91 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_92 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.716 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.716 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_93 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.745 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.745 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_94 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.764 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.765 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_95 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.793 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.793 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_96 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.821 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.822 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_97 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.850 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.850 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_98 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.872 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.872 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_99 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.894 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-24T18:15:25.894 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_100 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:25.917 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:270: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-24T18:15:25.917 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:270: TEST_divergent_ec: wait_for_clean 2026-02-24T18:15:25.917 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:15:25.918 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:15:25.918 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:15:25.918 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-24T18:15:25.918 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:15:25.918 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:15:25.918 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:15:25.919 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:15:25.919 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:15:25.988 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-24T18:15:25.988 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:15:25.988 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:15:25.988 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:15:25.988 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:15:25.988 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:15:26.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:15:26.204 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:15:26.204 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:15:26.204 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:15:26.204 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:15:26.204 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:15:26.291 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836493 2026-02-24T18:15:26.291 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836493 2026-02-24T18:15:26.291 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836493' 2026-02-24T18:15:26.292 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:15:26.292 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:15:26.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=42949672973 2026-02-24T18:15:26.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 42949672973 2026-02-24T18:15:26.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836493 1-42949672973' 2026-02-24T18:15:26.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:15:26.375 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:15:26.459 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=64424509452 2026-02-24T18:15:26.459 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 64424509452 2026-02-24T18:15:26.459 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836493 1-42949672973 2-64424509452' 2026-02-24T18:15:26.459 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:15:26.460 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836493 2026-02-24T18:15:26.460 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:15:26.460 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:15:26.462 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836493 2026-02-24T18:15:26.462 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:15:26.463 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836493 2026-02-24T18:15:26.463 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 21474836493 2026-02-24T18:15:26.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-24T18:15:26.464 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:15:26.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836491 -lt 21474836493 2026-02-24T18:15:26.685 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:15:27.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:15:27.687 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:15:27.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836493 -lt 21474836493 2026-02-24T18:15:27.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:15:27.899 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-42949672973 2026-02-24T18:15:27.899 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:15:27.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:15:27.901 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-42949672973 2026-02-24T18:15:27.901 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:15:27.903 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=42949672973 2026-02-24T18:15:27.903 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 42949672973' 2026-02-24T18:15:27.903 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 42949672973 2026-02-24T18:15:27.903 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:15:28.129 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 42949672973 -lt 42949672973 2026-02-24T18:15:28.129 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:15:28.130 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-64424509452 2026-02-24T18:15:28.130 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:15:28.131 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:15:28.132 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-64424509452 2026-02-24T18:15:28.132 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:15:28.133 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=64424509452 2026-02-24T18:15:28.133 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 64424509452 2026-02-24T18:15:28.134 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 64424509452' 2026-02-24T18:15:28.134 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:15:28.362 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 64424509452 -lt 64424509452 2026-02-24T18:15:28.362 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:15:28.363 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:15:28.363 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:15:28.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:15:28.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:15:28.658 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:15:28.658 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:15:28.658 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:15:28.658 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:15:28.658 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:15:28.659 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:15:28.877 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-24T18:15:28.878 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:15:28.878 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:15:28.878 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:15:29.173 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-24T18:15:29.173 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T18:15:29.173 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T18:15:29.173 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:272: TEST_divergent_ec: get_pg test existing_1 2026-02-24T18:15:29.173 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=test 2026-02-24T18:15:29.174 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=existing_1 2026-02-24T18:15:29.174 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map test existing_1 2026-02-24T18:15:29.174 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-24T18:15:29.400 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:272: TEST_divergent_ec: local pgid=1.0 2026-02-24T18:15:29.401 INFO:tasks.workunit.client.0.trial172.stdout:blackholing osds 0 2 2026-02-24T18:15:29.401 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:275: TEST_divergent_ec: echo 'blackholing osds 0 2' 2026-02-24T18:15:29.401 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:276: TEST_divergent_ec: ceph pg dump pgs 2026-02-24T18:15:29.608 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:15:29.608 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 0 0 0 51200 0 0 16 84 16 active+clean 2026-02-24T18:15:13.145460+0000 29'100 29:120 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:15:12.387431+0000 0'0 2026-02-24T18:15:12.387431+0000 0 0 periodic scrub scheduled @ 2026-02-25T22:44:02.327529+0000 0 0 2026-02-24T18:15:29.608 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:15:29.608 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:15:29.608 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:15:29.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:277: TEST_divergent_ec: for i in $non_divergent 2026-02-24T18:15:29.617 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:279: TEST_divergent_ec: get_asok_path osd.0 2026-02-24T18:15:29.617 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T18:15:29.617 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T18:15:29.618 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:15:29.618 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:15:29.618 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:15:29.618 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.0.asok 2026-02-24T18:15:29.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:279: TEST_divergent_ec: CEPH_ARGS= 2026-02-24T18:15:29.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:279: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-24T18:15:29.673 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:15:29.673 INFO:tasks.workunit.client.0.trial172.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-24T18:15:29.673 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:15:29.678 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:277: TEST_divergent_ec: for i in $non_divergent 2026-02-24T18:15:29.679 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:279: TEST_divergent_ec: get_asok_path osd.2 2026-02-24T18:15:29.679 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-24T18:15:29.679 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-24T18:15:29.679 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:15:29.679 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:15:29.679 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:15:29.680 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.2.asok 2026-02-24T18:15:29.680 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:279: TEST_divergent_ec: CEPH_ARGS= 2026-02-24T18:15:29.680 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:279: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-24T18:15:29.732 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:15:29.732 INFO:tasks.workunit.client.0.trial172.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-24T18:15:29.732 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:15:29.738 INFO:tasks.workunit.client.0.trial172.stdout:writing divergent object 2026-02-24T18:15:29.738 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:283: TEST_divergent_ec: echo 'writing divergent object' 2026-02-24T18:15:29.738 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:285: TEST_divergent_ec: sleep 1 2026-02-24T18:15:29.738 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:284: TEST_divergent_ec: rados -p test put existing_100 /tmp/tmp.ltmCGn22fu 2026-02-24T18:15:30.740 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:287: TEST_divergent_ec: rados -p test mksnap snap1 2026-02-24T18:15:30.741 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:286: TEST_divergent_ec: rados -p test put existing_100 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:30.813 INFO:tasks.workunit.client.0.trial172.stdout:created pool test snap snap1 2026-02-24T18:15:30.815 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:289: TEST_divergent_ec: sleep 10 2026-02-24T18:15:30.816 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:288: TEST_divergent_ec: expr 100 - 1 2026-02-24T18:15:30.817 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:288: TEST_divergent_ec: rados -p test put existing_99 /tmp/tmp.oW6cFhP1GX 2026-02-24T18:15:40.817 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:290: TEST_divergent_ec: killall -9 rados 2026-02-24T18:15:40.865 INFO:tasks.workunit.client.0.trial172.stderr:rados: no process found 2026-02-24T18:15:40.865 INFO:tasks.workunit.client.0.trial172.stdout:killing all the osds 2026-02-24T18:15:40.866 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:293: TEST_divergent_ec: echo 'killing all the osds' 2026-02-24T18:15:40.866 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:294: TEST_divergent_ec: ceph pg dump pgs 2026-02-24T18:15:41.081 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:15:41.081 INFO:tasks.workunit.client.0.trial172.stdout:1.0 101 0 0 0 0 51712 0 0 14 90 14 active+clean 2026-02-24T18:15:13.145460+0000 30'104 30:126 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:15:12.387431+0000 0'0 2026-02-24T18:15:12.387431+0000 0 0 periodic scrub scheduled @ 2026-02-26T04:05:43.846181+0000 0 0 2026-02-24T18:15:41.081 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:15:41.081 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:15:41.082 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:15:41.090 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:295: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd 2026-02-24T18:15:41.090 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:15:41.091 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:15:41.091 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:15:41.091 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:15:41.091 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:15:41.200 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:15:41.200 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:296: TEST_divergent_ec: for i in $osds 2026-02-24T18:15:41.200 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:298: TEST_divergent_ec: ceph osd down osd.0 2026-02-24T18:15:41.469 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already down. 2026-02-24T18:15:41.479 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:296: TEST_divergent_ec: for i in $osds 2026-02-24T18:15:41.480 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:298: TEST_divergent_ec: ceph osd down osd.1 2026-02-24T18:15:41.759 INFO:tasks.workunit.client.0.trial172.stderr:osd.1 is already down. 2026-02-24T18:15:41.770 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:296: TEST_divergent_ec: for i in $osds 2026-02-24T18:15:41.770 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:298: TEST_divergent_ec: ceph osd down osd.2 2026-02-24T18:15:42.037 INFO:tasks.workunit.client.0.trial172.stderr:osd.2 is already down. 2026-02-24T18:15:42.048 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:300: TEST_divergent_ec: for i in $non_divergent 2026-02-24T18:15:42.048 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:302: TEST_divergent_ec: ceph osd out osd.0 2026-02-24T18:15:42.342 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already out. 2026-02-24T18:15:42.352 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:300: TEST_divergent_ec: for i in $non_divergent 2026-02-24T18:15:42.352 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:302: TEST_divergent_ec: ceph osd out osd.2 2026-02-24T18:15:42.639 INFO:tasks.workunit.client.0.trial172.stderr:osd.2 is already out. 2026-02-24T18:15:42.649 INFO:tasks.workunit.client.0.trial172.stdout:bringing up non_divergent 0 2 2026-02-24T18:15:42.649 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:306: TEST_divergent_ec: echo 'bringing up non_divergent 0 2' 2026-02-24T18:15:42.649 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:307: TEST_divergent_ec: ceph pg dump pgs 2026-02-24T18:15:42.859 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:15:42.859 INFO:tasks.workunit.client.0.trial172.stdout:1.0 101 0 0 0 0 51712 0 0 14 90 14 stale+active+clean 2026-02-24T18:15:13.145460+0000 30'104 30:126 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:15:12.387431+0000 0'0 2026-02-24T18:15:12.387431+0000 0 0 periodic scrub scheduled @ 2026-02-26T04:05:43.846181+0000 0 0 2026-02-24T18:15:42.860 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:15:42.860 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:15:42.860 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:15:42.871 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:308: TEST_divergent_ec: for i in $non_divergent 2026-02-24T18:15:42.871 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:310: TEST_divergent_ec: activate_osd td/divergent-priors 0 2026-02-24T18:15:42.871 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:15:42.872 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:15:42.872 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-24T18:15:42.872 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:15:42.872 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/0 2026-02-24T18:15:42.872 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:15:42.872 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:15:42.872 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:15:42.872 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:15:42.872 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-24T18:15:42.872 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-24T18:15:42.872 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:15:42.873 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:15:42.873 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:15:42.873 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:15:42.873 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:15:42.873 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:15:42.873 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:15:42.873 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:15:42.873 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:15:42.873 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:15:42.873 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:15:42.873 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:15:42.873 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:15:42.874 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:15:42.874 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:15:42.874 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:15:42.874 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:15:42.874 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:15:42.874 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:15:42.874 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:15:42.874 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/0 2026-02-24T18:15:42.875 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:15:42.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-24T18:15:42.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:15:42.876 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/0/whoami 2026-02-24T18:15:42.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-24T18:15:42.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:15:42.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:15:42.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:15:42.899 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:42.894+0000 7fc42d762a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:42.901 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:42.894+0000 7fc42d762a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:42.902 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:42.894+0000 7fc42d762a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:43.097 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-24T18:15:43.097 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:15:43.097 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:15:43.097 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:15:43.097 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:15:43.097 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:15:43.098 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:15:43.098 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:15:43.099 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:15:43.099 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:15:43.321 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:15:43.364 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:43.358+0000 7fc42d762a40 -1 Falling back to public interface 2026-02-24T18:15:44.033 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:44.026+0000 7fc42d762a40 -1 osd.0 29 log_to_monitors true 2026-02-24T18:15:44.323 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:15:44.323 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:15:44.323 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:15:44.323 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:15:44.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:15:44.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:15:44.538 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up out weight 0 up_from 37 up_thru 0 down_at 31 last_clean_interval [5,29) [v2:127.0.0.1:6800/349667324,v1:127.0.0.1:6801/349667324] [v2:127.0.0.1:6802/349667324,v1:127.0.0.1:6803/349667324] exists,up 93b13c7c-e44d-45ef-876f-6c2e68859b9a 2026-02-24T18:15:44.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:15:44.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:15:44.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:15:44.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:308: TEST_divergent_ec: for i in $non_divergent 2026-02-24T18:15:44.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:310: TEST_divergent_ec: activate_osd td/divergent-priors 2 2026-02-24T18:15:44.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:15:44.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:15:44.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T18:15:44.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:15:44.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/2 2026-02-24T18:15:44.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:15:44.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:15:44.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:15:44.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:15:44.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-24T18:15:44.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-24T18:15:44.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:15:44.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:15:44.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:15:44.540 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:15:44.540 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:15:44.540 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:15:44.540 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:15:44.540 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:15:44.540 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:15:44.540 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:15:44.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:15:44.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:15:44.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:15:44.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:15:44.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:15:44.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:15:44.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:15:44.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:15:44.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:15:44.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:15:44.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/2 2026-02-24T18:15:44.542 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:15:44.542 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T18:15:44.542 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:15:44.543 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/2/whoami 2026-02-24T18:15:44.544 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T18:15:44.544 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:15:44.544 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:15:44.544 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:15:44.568 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:44.562+0000 7f79d20efa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:44.569 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:44.562+0000 7f79d20efa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:44.570 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:44.566+0000 7f79d20efa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:15:44.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T18:15:44.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:15:44.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:15:44.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:15:44.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:15:44.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:15:44.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:15:44.762 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:15:44.762 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:15:44.762 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:15:44.948 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:44.942+0000 7f79d20efa40 -1 Falling back to public interface 2026-02-24T18:15:44.984 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:15:45.552 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:15:45.546+0000 7f79d20efa40 -1 osd.2 29 log_to_monitors true 2026-02-24T18:15:45.985 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:15:45.986 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:15:45.986 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:15:45.987 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:15:45.987 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:15:45.987 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:15:46.202 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up out weight 0 up_from 39 up_thru 0 down_at 33 last_clean_interval [15,29) [v2:127.0.0.1:6810/560661905,v1:127.0.0.1:6811/560661905] [v2:127.0.0.1:6812/560661905,v1:127.0.0.1:6813/560661905] exists,up 4e270c3f-1799-484f-8fb2-c5e0604b5c26 2026-02-24T18:15:46.202 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:15:46.202 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:15:46.202 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:15:46.202 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:312: TEST_divergent_ec: for i in $non_divergent 2026-02-24T18:15:46.202 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:314: TEST_divergent_ec: ceph osd in osd.0 2026-02-24T18:15:46.475 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already in. 2026-02-24T18:15:46.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:312: TEST_divergent_ec: for i in $non_divergent 2026-02-24T18:15:46.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:314: TEST_divergent_ec: ceph osd in osd.2 2026-02-24T18:15:46.767 INFO:tasks.workunit.client.0.trial172.stderr:osd.2 is already in. 2026-02-24T18:15:46.776 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:317: TEST_divergent_ec: sleep 5 2026-02-24T18:15:51.779 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:321: TEST_divergent_ec: expr 2 + 2 2026-02-24T18:15:51.780 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:321: TEST_divergent_ec: objname=existing_4 2026-02-24T18:15:51.780 INFO:tasks.workunit.client.0.trial172.stdout:writing non-divergent object existing_4 2026-02-24T18:15:51.781 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:322: TEST_divergent_ec: echo 'writing non-divergent object existing_4' 2026-02-24T18:15:51.781 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:323: TEST_divergent_ec: ceph pg dump pgs 2026-02-24T18:15:51.987 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:15:51.988 INFO:tasks.workunit.client.0.trial172.stdout:1.0 100 0 100 0 0 51200 0 0 16 84 16 active+undersized+degraded 2026-02-24T18:15:46.812358+0000 29'100 42:130 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-24T18:15:12.387431+0000 0'0 2026-02-24T18:15:12.387431+0000 0 0 periodic scrub scheduled @ 2026-02-25T18:56:43.317373+0000 0 0 2026-02-24T18:15:51.988 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:15:51.988 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:15:51.988 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:15:51.997 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:324: TEST_divergent_ec: rados -p test put existing_4 /tmp/tmp.ltmCGn22fu 2026-02-24T18:15:52.028 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:325: TEST_divergent_ec: rm -f /tmp/tmp.ltmCGn22fu 2026-02-24T18:15:52.029 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:327: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-24T18:15:52.029 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:327: TEST_divergent_ec: wait_for_clean 2026-02-24T18:15:52.029 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:15:52.029 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:15:52.029 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:15:52.030 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-24T18:15:52.030 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:15:52.030 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:15:52.031 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:15:52.031 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:15:52.031 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:15:52.101 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-24T18:15:52.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:15:52.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:15:52.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:15:52.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:15:52.102 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:15:52.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:15:52.324 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:15:52.324 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:15:52.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:15:52.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:15:52.324 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:15:52.408 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=158913789956 2026-02-24T18:15:52.408 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 158913789956 2026-02-24T18:15:52.408 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-158913789956' 2026-02-24T18:15:52.408 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:15:52.408 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:15:52.472 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq= 2026-02-24T18:15:52.472 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: true 2026-02-24T18:15:52.472 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z '' 2026-02-24T18:15:52.472 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: flush_pg_stats: continue 2026-02-24T18:15:52.472 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:15:52.472 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:15:52.554 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=167503724548 2026-02-24T18:15:52.554 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 167503724548 2026-02-24T18:15:52.554 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-158913789956 2-167503724548' 2026-02-24T18:15:52.554 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:15:52.555 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-158913789956 2026-02-24T18:15:52.555 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:15:52.556 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:15:52.557 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-158913789956 2026-02-24T18:15:52.557 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:15:52.559 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=158913789956 2026-02-24T18:15:52.559 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 158913789956 2026-02-24T18:15:52.559 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 158913789956' 2026-02-24T18:15:52.560 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:15:52.780 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789954 -lt 158913789956 2026-02-24T18:15:52.780 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:15:53.781 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:15:53.782 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:15:54.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789956 -lt 158913789956 2026-02-24T18:15:54.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:15:54.006 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-167503724548 2026-02-24T18:15:54.006 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:15:54.008 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:15:54.008 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-167503724548 2026-02-24T18:15:54.008 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:15:54.010 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=167503724548 2026-02-24T18:15:54.010 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 167503724548 2026-02-24T18:15:54.010 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 167503724548' 2026-02-24T18:15:54.010 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:15:54.234 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 167503724548 -lt 167503724548 2026-02-24T18:15:54.235 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:15:54.235 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:15:54.235 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:15:54.529 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:15:54.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:15:54.530 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:15:54.530 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:15:54.530 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:15:54.530 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:15:54.530 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:15:54.530 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:15:54.749 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:15:54.749 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:15:54.749 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:15:54.749 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:15:55.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:15:55.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-24T18:15:55.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-24T18:15:55.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-24T18:15:55.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:15:55.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-24T18:15:55.140 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:15:55.140 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:15:55.141 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:15:55.141 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:15:55.141 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:15:55.141 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:15:55.141 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:15:55.141 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:15:55.354 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:15:55.355 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:15:55.355 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:15:55.356 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:15:55.653 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:15:55.654 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:15:55.654 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:15:55.654 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:15:55.654 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:15:55.654 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:15:55.654 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:15:55.654 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:15:55.655 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:15:55.947 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:15:55.947 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:15:55.947 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 8 )) 2026-02-24T18:15:55.947 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:15:55.947 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-24T18:15:56.147 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:15:56.148 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:15:56.149 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:15:56.149 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:15:56.149 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:15:56.149 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:15:56.149 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:15:56.149 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:15:56.365 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:15:56.366 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:15:56.366 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:15:56.366 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:15:56.657 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:15:56.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:15:56.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:15:56.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:15:56.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:15:56.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:15:56.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:15:56.658 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:15:56.658 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:15:56.950 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:15:56.950 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:15:56.951 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 8 )) 2026-02-24T18:15:56.951 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:15:56.951 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-24T18:15:57.351 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:15:57.351 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:15:57.352 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:15:57.353 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:15:57.353 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:15:57.353 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:15:57.353 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:15:57.353 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:15:57.573 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:15:57.573 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:15:57.574 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:15:57.574 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:15:57.864 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:15:57.865 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:15:57.865 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:15:57.865 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:15:57.865 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:15:57.865 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:15:57.865 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:15:57.865 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:15:57.865 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:15:58.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:15:58.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:15:58.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 8 )) 2026-02-24T18:15:58.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:15:58.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-24T18:15:58.951 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:15:58.952 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:15:58.953 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:15:58.953 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:15:58.953 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:15:58.953 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:15:58.953 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:15:58.953 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:15:59.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:15:59.165 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:15:59.165 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:15:59.165 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:15:59.444 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:15:59.444 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:15:59.444 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:15:59.444 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:15:59.444 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:15:59.444 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:15:59.444 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:15:59.444 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:15:59.444 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:15:59.737 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:15:59.737 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:15:59.737 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 8 )) 2026-02-24T18:15:59.737 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:15:59.738 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 1.6 2026-02-24T18:16:01.339 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:16:01.339 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:16:01.340 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:16:01.340 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:16:01.340 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:16:01.341 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:16:01.341 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:16:01.341 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:16:01.553 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:16:01.554 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:16:01.554 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:16:01.554 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:16:01.838 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:16:01.838 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:16:01.839 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:16:01.839 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:16:01.839 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:16:01.839 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:16:01.839 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:16:01.839 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:16:01.839 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:16:02.130 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:16:02.130 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:16:02.130 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 5 >= 8 )) 2026-02-24T18:16:02.131 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:16:02.131 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 3.2 2026-02-24T18:16:05.331 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:16:05.331 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:16:05.332 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:16:05.332 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:16:05.332 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:16:05.333 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:16:05.333 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:16:05.333 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:16:05.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:16:05.541 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:16:05.541 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:16:05.541 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:16:05.832 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:16:05.833 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:16:05.833 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:16:05.833 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:16:05.833 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:16:05.833 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:16:05.833 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:16:05.833 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:16:05.833 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:16:06.134 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:16:06.134 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:16:06.134 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 6 >= 8 )) 2026-02-24T18:16:06.134 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:16:06.134 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 6.4 2026-02-24T18:16:12.535 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:16:12.535 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:16:12.536 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:16:12.536 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:16:12.536 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:16:12.537 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:16:12.537 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:16:12.537 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:16:12.758 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:16:12.759 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:16:12.759 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:16:12.759 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:16:13.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:16:13.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:16:13.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:16:13.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:16:13.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:16:13.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:16:13.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:16:13.051 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:16:13.051 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:16:13.336 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:16:13.336 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:16:13.336 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 7 >= 8 )) 2026-02-24T18:16:13.337 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:16:13.337 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 7.3 2026-02-24T18:16:20.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:16:20.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:16:20.639 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:16:20.639 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:16:20.639 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:16:20.640 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:16:20.640 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:16:20.640 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:16:20.849 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:16:20.849 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:16:20.850 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:16:20.850 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:16:21.132 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:16:21.132 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:16:21.132 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:16:21.132 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:16:21.132 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:16:21.132 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:16:21.133 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:16:21.133 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:16:21.133 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:16:21.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:16:21.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:16:21.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 8 >= 8 )) 2026-02-24T18:16:21.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-24T18:16:21.698 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_fingerprint": "70393c9b-3fd2-4a6c-9859-cb6a60af12dc", 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "version": "20.3.0-5507-g618c318a", 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "commit": "618c318a7a0b287ab5aeff207b7bf1d73ef45044", 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "timestamp": "2026-02-24T18:16:21.696329+0000", 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "tag": "", 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "health": { 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "status": "HEALTH_WARN", 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "checks": { 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "OSDMAP_FLAGS": { 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "count": 17 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [], 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.699 INFO:tasks.workunit.client.0.trial172.stdout: "OSD_DOWN": { 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: "message": "1 osds down", 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: "count": 1 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [ 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: "message": "osd.1 (root=default,host=trial172) is down" 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: "PG_DEGRADED": { 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: "message": "Degraded data redundancy: 101/303 objects degraded (33.333%), 1 pg degraded", 2026-02-24T18:16:21.700 INFO:tasks.workunit.client.0.trial172.stdout: "count": 1 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [ 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [NONE,0,2]" 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: "mutes": [] 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: "monmap_first_committed": 1, 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: "monmap_last_committed": 1, 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: "monmap": { 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 1, 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: "fsid": "27512777-e37f-4f16-b176-1621842bd4df", 2026-02-24T18:16:21.701 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:14:58.797613Z", 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "created": "2026-02-24T18:14:58.797613Z", 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "min_mon_release": 20, 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "min_mon_release_name": "tentacle", 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "election_strategy": 1, 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "disallowed_leaders": "", 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode": false, 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "tiebreaker_mon": "", 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "removed_ranks": "", 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "features": { 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "persistent": [ 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "kraken", 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "luminous", 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "mimic", 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap-prune", 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "nautilus", 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "octopus", 2026-02-24T18:16:21.702 INFO:tasks.workunit.client.0.trial172.stdout: "pacific", 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: "elector-pinging", 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: "quincy", 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: "reef", 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: "squid", 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: "tentacle", 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: "nvmeof_beacon_diff" 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: "optional": [] 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: "mons": [ 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: "rank": 0, 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: "name": "a", 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:7115", 2026-02-24T18:16:21.703 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 0 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:7115/0", 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: "priority": 0, 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 0, 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: "time_added": "2026-02-24T18:14:58:797607+0000", 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: "crush_location": [] 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: "quorum": [ 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: 0 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap": { 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 42, 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: "fsid": "27512777-e37f-4f16-b176-1621842bd4df", 2026-02-24T18:16:21.704 INFO:tasks.workunit.client.0.trial172.stdout: "created": "2026-02-24T18:14:58.882667+0000", 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:15:46.809731+0000", 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "last_up_change": "2026-02-24T18:15:45.705801+0000", 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "last_in_change": "2026-02-24T18:15:46.707383+0000", 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "flags_num": 5800384, 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "flags_set": [ 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "nodown", 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "noin", 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "noout", 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "pglog_hardlimit", 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snapdirs", 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "recovery_deletes", 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "sortbitwise" 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "crush_version": 10, 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "full_ratio": 0.99000000953674316, 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-24T18:16:21.705 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_snapshot": "", 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "pool_max": 1, 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "max_osd": 3, 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "require_min_compat_client": "luminous", 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "min_compat_client": "jewel", 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "require_osd_release": "tentacle", 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "allow_crimson": false, 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "pools": [ 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "pool": 1, 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "pool_name": "test", 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "create_time": "2026-02-24T18:15:12.294587+0000", 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "flags": 16389, 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "flags_names": "hashpspool,ec_overwrites,pool_snaps", 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "type": 3, 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "size": 3, 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "min_size": 2, 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "crush_rule": 1, 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_count": 0, 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_target": 0, 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_barrier": 0, 2026-02-24T18:16:21.706 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "is_stretch_pool": false, 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "object_hash": 2, 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "pg_autoscale_mode": "off", 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num": 1, 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "pg_placement_num": 1, 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "pg_placement_num_target": 1, 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num_target": 1, 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num_pending": 1, 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "last_pg_merge_meta": { 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "source_pgid": "0.0", 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "ready_epoch": 0, 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_started": 0, 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_clean": 0, 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "source_version": "0'0", 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "target_version": "0'0" 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "last_change": "30", 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend": "0", 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend_preluminous": "0", 2026-02-24T18:16:21.707 INFO:tasks.workunit.client.0.trial172.stdout: "auid": 0, 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "snap_mode": "pool", 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "snap_seq": 1, 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "snap_epoch": 30, 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "pool_snaps": [ 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "snapid": 1, 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "stamp": "2026-02-24T18:15:30.759768+0000", 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "name": "snap1" 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "removed_snaps": "[]", 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "quota_max_bytes": 0, 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "quota_max_objects": 0, 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "tiers": [], 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "tier_of": -1, 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "read_tier": -1, 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "write_tier": -1, 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "cache_mode": "none", 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "target_max_bytes": 0, 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "target_max_objects": 0, 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-24T18:16:21.708 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "cache_min_flush_age": 0, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "cache_min_evict_age": 0, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "erasure_code_profile": "myprofile", 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_params": { 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "type": "none" 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_period": 0, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_count": 0, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "use_gmt_hitset": true, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "min_read_recency_for_promote": 0, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "min_write_recency_for_promote": 0, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_grade_decay_rate": 0, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_search_last_n": 0, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "grade_table": [], 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "stripe_width": 8192, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "expected_num_objects": 0, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "fast_read": false, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "nonprimary_shards": "{}", 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "options": {}, 2026-02-24T18:16:21.709 INFO:tasks.workunit.client.0.trial172.stdout: "application_metadata": {} 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "osds": [ 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "93b13c7c-e44d-45ef-876f-6c2e68859b9a", 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "up": 1, 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 5, 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 29, 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 37, 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 41, 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 31, 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:21.710 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6800", 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 349667324 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6801", 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 349667324 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6802", 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 349667324 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6803", 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 349667324 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.711 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6806", 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 349667324 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6807", 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 349667324 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6804", 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 349667324 2026-02-24T18:16:21.712 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6805", 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 349667324 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6801/349667324", 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6803/349667324", 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6807/349667324", 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6805/349667324", 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "exists", 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "up" 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "a1643499-a87c-4291-842c-c57767e846a8", 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "up": 0, 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:16:21.713 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 0, 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 0, 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 10, 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 25, 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 32, 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6810", 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6811", 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.714 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6812", 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6813", 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6816", 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6817", 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.715 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6814", 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6815", 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6811/427574815", 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6813/427574815", 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6817/427574815", 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6815/427574815", 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "exists" 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "4e270c3f-1799-484f-8fb2-c5e0604b5c26", 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "up": 1, 2026-02-24T18:16:21.716 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 15, 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 29, 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 39, 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 0, 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 33, 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6810", 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 560661905 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6811", 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 560661905 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:16:21.717 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6812", 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 560661905 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6813", 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 560661905 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6816", 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 560661905 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6817", 2026-02-24T18:16:21.718 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 560661905 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6814", 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 560661905 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6815", 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 560661905 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6811/560661905", 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6813/560661905", 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6817/560661905", 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6815/560661905", 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: "exists", 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: "up" 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.719 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "osd_xinfo": [ 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:15:41.409648+0000", 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:15:02.261828+0000", 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:15:41.701782+0000", 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:15:06.087865+0000", 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:15:41.980195+0000", 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:16:21.720 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:15:10.037025+0000", 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap": [], 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap_items": [], 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap_primaries": [], 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "pg_temp": [], 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "primary_temp": [], 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "blocklist": {}, 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "range_blocklist": {}, 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "erasure_code_profiles": { 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "default": { 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "k": "2", 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "m": "2", 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "plugin": "isa", 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "technique": "reed_sol_van" 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "myprofile": { 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "crush-device-class": "", 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "crush-failure-domain": "osd", 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "crush-num-failure-domains": "0", 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "crush-osds-per-failure-domain": "0", 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "crush-root": "default", 2026-02-24T18:16:21.721 INFO:tasks.workunit.client.0.trial172.stdout: "k": "2", 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "m": "1", 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "plugin": "isa", 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "technique": "reed_sol_van" 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "removed_snaps_queue": [], 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "new_removed_snaps": [], 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "new_purged_snaps": [], 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "crush_node_flags": {}, 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "device_class_flags": {}, 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode": { 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode_enabled": false, 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_bucket_count": 0, 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "degraded_stretch_mode": 0, 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "recovering_stretch_mode": 0, 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode_bucket": 0 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "osd_metadata": [ 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6802/349667324,v1:127.0.0.1:6803/349667324]", 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:16:21.722 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:14:59.857538Z", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:16:21.723 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6800/349667324,v1:127.0.0.1:6801/349667324]", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/349667324,v1:127.0.0.1:6807/349667324]", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/349667324,v1:127.0.0.1:6805/349667324]", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/0", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6812/427574815,v1:127.0.0.1:6813/427574815]", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:16:21.724 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:15:03.492854Z", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:16:21.725 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6810/427574815,v1:127.0.0.1:6811/427574815]", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/427574815,v1:127.0.0.1:6817/427574815]", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/427574815,v1:127.0.0.1:6815/427574815]", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/1", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6812/560661905,v1:127.0.0.1:6813/560661905]", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:16:21.726 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:15:07.600471Z", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6810/560661905,v1:127.0.0.1:6811/560661905]", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/560661905,v1:127.0.0.1:6817/560661905]", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/560661905,v1:127.0.0.1:6815/560661905]", 2026-02-24T18:16:21.727 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/2", 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_clean_epochs": { 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "min_last_epoch_clean": 0, 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_clean": { 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "per_pool": [] 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "osd_epochs": [ 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 37 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 40 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.728 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_first_committed": 1, 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_last_committed": 42, 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "crushmap": { 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "devices": [ 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.0", 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.1", 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.2", 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "types": [ 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 0, 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd" 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "name": "host" 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 2, 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: "name": "chassis" 2026-02-24T18:16:21.729 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 3, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "name": "rack" 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 4, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "name": "row" 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 5, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "name": "pdu" 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 6, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "name": "pod" 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 7, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "name": "room" 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 8, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "name": "datacenter" 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 9, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "name": "zone" 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 10, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: "name": "region" 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.730 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "name": "root" 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "buckets": [ 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "id": -1, 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "name": "default", 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "root", 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "id": -3, 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "id": -2, 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "name": "default~ssd", 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "root", 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "id": -4, 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:16:21.731 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "id": -3, 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "name": "trial172", 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "host", 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 1 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 2 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "id": -4, 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "name": "trial172~ssd", 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "host", 2026-02-24T18:16:21.732 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 1 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 2 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "rules": [ 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "rule_id": 0, 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "rule_name": "replicated_rule", 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "type": 1, 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "steps": [ 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "op": "take", 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "item": -1, 2026-02-24T18:16:21.733 INFO:tasks.workunit.client.0.trial172.stdout: "item_name": "default" 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: "op": "choose_firstn", 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: "num": 0, 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: "type": "osd" 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: "op": "emit" 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: "rule_id": 1, 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: "rule_name": "test", 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: "type": 3, 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: "steps": [ 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: "op": "set_chooseleaf_tries", 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: "num": 5 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.734 INFO:tasks.workunit.client.0.trial172.stderr:report 1530617232 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "op": "set_choose_tries", 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "num": 100 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "op": "take", 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "item": -1, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "item_name": "default" 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "op": "choose_indep", 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "num": 0, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "type": "osd" 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "op": "emit" 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "tunables": { 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "choose_local_tries": 0, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "choose_local_fallback_tries": 0, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "choose_total_tries": 50, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_descend_once": 1, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_vary_r": 1, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_stable": 1, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "msr_descents": 100, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "msr_collision_tries": 100, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "straw_calc_version": 1, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "allowed_bucket_algs": 54, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "profile": "jewel", 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "optimal_tunables": 1, 2026-02-24T18:16:21.735 INFO:tasks.workunit.client.0.trial172.stdout: "legacy_tunables": 0, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "minimum_required_version": "jewel", 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables": 1, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables2": 1, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "has_v2_rules": 1, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables3": 1, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "has_v3_rules": 0, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "has_v4_buckets": 1, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables5": 1, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "has_v5_rules": 0, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "has_msr_rules": 0 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "choose_args": {} 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "fsmap": { 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 1, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "btime": "2026-02-24T18:14:58:882446+0000", 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "default_fscid": -1, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "compat": { 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "compat": {}, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "ro_compat": {}, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "incompat": { 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "feature_1": "base v0.20", 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "feature_2": "client writeable ranges", 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "feature_3": "default file layouts on dirs", 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "feature_4": "dir inode in separate object", 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "feature_5": "mds uses versioned encoding", 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "feature_8": "no anchor table", 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "feature_9": "file layout v2", 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "feature_10": "snaprealm v2", 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "feature_11": "minor log segments", 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "feature_12": "quiesce subvolumes" 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "feature_flags": { 2026-02-24T18:16:21.736 INFO:tasks.workunit.client.0.trial172.stdout: "enable_multiple": true, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "ever_enabled_multiple": true 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "standbys": [], 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "filesystems": [] 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "mdsmap_first_committed": 1, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "mdsmap_last_committed": 1, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "auth": { 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "first_committed": 1, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "last_committed": 5, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_secrets": 3 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg": 1, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_active": 1, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_unknown": 0, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_osd": 3, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "pool_sum": { 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "stat_sum": { 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes": 52200, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects": 101, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_clones": 1, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_copies": 303, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing": 0, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_degraded": 101, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_misplaced": 0, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_unfound": 0, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_dirty": 101, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_whiteouts": 0, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_read": 0, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_read_kb": 0, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_write": 101, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_write_kb": 101, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_scrub_errors": 0, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_deep_scrub_errors": 0, 2026-02-24T18:16:21.737 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_recovered": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_recovered": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_keys_recovered": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_omap": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_kb": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_kb": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_promote": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_high": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_low": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_some": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_full": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_pinned": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_legacy_snapsets": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_large_omap_objects": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_manifest": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_bytes": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_keys": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_repaired": 0 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "store_stats": { 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "total": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "available": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 0, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 0 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "log_size": 18, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "ondisk_log_size": 18, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "up": 3, 2026-02-24T18:16:21.738 INFO:tasks.workunit.client.0.trial172.stdout: "acting": 3, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "num_store_stats": 0 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "osd_sum": { 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 0, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 0, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "num_pgs": 3, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "num_osds": 3, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_osds": 3, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_omap_osds": 3, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "kb": 314572800, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used": 83924, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_data": 2148, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_omap": 27, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_meta": 81700, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "kb_avail": 314488876, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "statfs": { 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "total": 322122547200, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "available": 322036609024, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 2199552, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 1540753, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 28369, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 83661103 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "hb_peers": [], 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "snap_trim_queue_len": 0, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "num_snap_trimming": 0, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "num_shards_repaired": 0, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "op_queue_age_hist": { 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "histogram": [], 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "upper_bound": 1 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "perf_stat": { 2026-02-24T18:16:21.739 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ms": 0, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ms": 0, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ns": 0, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ns": 0 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "alerts": [], 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "network_ping_times": [] 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "osd_sum_by_class": { 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "ssd": { 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 0, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 0, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "num_pgs": 3, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "num_osds": 3, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_osds": 3, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_omap_osds": 3, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "kb": 314572800, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used": 83924, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_data": 2148, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_omap": 27, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_meta": 81700, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "kb_avail": 314488876, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "statfs": { 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "total": 322122547200, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "available": 322036609024, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 2199552, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 1540753, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 28369, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 83661103 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "hb_peers": [], 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "snap_trim_queue_len": 0, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "num_snap_trimming": 0, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "num_shards_repaired": 0, 2026-02-24T18:16:21.740 INFO:tasks.workunit.client.0.trial172.stdout: "op_queue_age_hist": { 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "histogram": [], 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "upper_bound": 1 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "perf_stat": { 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ms": 0, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ms": 0, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ns": 0, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ns": 0 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "alerts": [], 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "network_ping_times": [] 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "pool_stats": [ 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": 1, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg": 1, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "stat_sum": { 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes": 52200, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects": 101, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_clones": 1, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_copies": 303, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing": 0, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_degraded": 101, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_misplaced": 0, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_unfound": 0, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_dirty": 101, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_whiteouts": 0, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_read": 0, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_read_kb": 0, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_write": 101, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_write_kb": 101, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_scrub_errors": 0, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_deep_scrub_errors": 0, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_recovered": 0, 2026-02-24T18:16:21.741 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_recovered": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_keys_recovered": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_omap": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_kb": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_kb": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_promote": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_high": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_low": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_some": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_full": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_pinned": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_legacy_snapsets": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_large_omap_objects": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_manifest": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_bytes": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_keys": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_repaired": 0 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "store_stats": { 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "total": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "available": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 1241088, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 1241088, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 0, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 0 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "log_size": 18, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "ondisk_log_size": 18, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "up": 3, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "acting": 3, 2026-02-24T18:16:21.742 INFO:tasks.workunit.client.0.trial172.stdout: "num_store_stats": 3 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "osd_stats": [ 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 158913789962 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 42949672975 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 167503724554 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_by_state": [ 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "state": "active+undersized+degraded", 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "num": 1 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "pool_pg_unavailable_map": [ 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": "1", 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "pgs": [] 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_by_osd": [ 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2147483647, 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 1, 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 1, 2026-02-24T18:16:21.743 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 1, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 1, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 0, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snaps": [ 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "pool": 1, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snaps": {} 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "servicemap": { 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 3, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:16:07.341425+0000", 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "services": { 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "osd": { 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "daemons": { 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "summary": "", 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "0": { 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "start_epoch": 0, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "start_stamp": "0.000000", 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "gid": 0, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "metadata": {}, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "task_status": {} 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "2": { 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "start_epoch": 0, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "start_stamp": "0.000000", 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "gid": 0, 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-24T18:16:21.744 INFO:tasks.workunit.client.0.trial172.stdout: "metadata": {}, 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: "task_status": {} 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: "mgrstat_first_committed": 1, 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: "mgrstat_last_committed": 42, 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: "logm_first_committed": 1, 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: "logm_last_committed": 234, 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: "paxos": { 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: "first_committed": 1, 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: "last_committed": 328, 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: "last_pn": 0, 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: "accepted_pn": 0 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:330: TEST_divergent_ec: for i in $non_divergent 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:332: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.0 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:16:21.745 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:16:21.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:16:21.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:333: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 0 --op log --pgid 1.0 2026-02-24T18:16:21.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-24T18:16:21.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T18:16:21.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-24T18:16:21.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T18:16:21.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/divergent-priors/0 2026-02-24T18:16:21.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/0 --op log --pgid 1.0 2026-02-24T18:16:21.988 INFO:tasks.workunit.client.0.trial172.stderr:PG '1.0' not found 2026-02-24T18:16:22.084 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1256: _objectstore_tool_nodown: return 1 2026-02-24T18:16:22.084 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:334: TEST_divergent_ec: activate_osd td/divergent-priors 0 2026-02-24T18:16:22.084 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:16:22.084 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:16:22.084 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/0 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:16:22.085 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:16:22.086 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:16:22.086 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:16:22.086 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:16:22.086 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:16:22.087 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:16:22.087 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:16:22.087 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:16:22.087 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:16:22.087 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:16:22.087 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:16:22.087 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:16:22.087 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/0 2026-02-24T18:16:22.087 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:16:22.088 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-24T18:16:22.088 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:16:22.088 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/0/whoami 2026-02-24T18:16:22.088 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-24T18:16:22.088 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:16:22.088 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:16:22.088 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:16:22.111 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:16:22.105+0000 7ff9feab2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:16:22.113 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:16:22.105+0000 7ff9feab2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:16:22.114 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:16:22.105+0000 7ff9feab2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:16:22.315 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-24T18:16:22.315 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:16:22.315 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:16:22.315 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:16:22.315 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:16:22.315 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:16:22.315 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:16:22.315 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:16:22.315 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:16:22.316 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:16:22.505 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:16:22.497+0000 7ff9feab2a40 -1 Falling back to public interface 2026-02-24T18:16:22.537 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 37 up_thru 41 down_at 31 last_clean_interval [5,29) [v2:127.0.0.1:6800/349667324,v1:127.0.0.1:6801/349667324] [v2:127.0.0.1:6802/349667324,v1:127.0.0.1:6803/349667324] exists,up 93b13c7c-e44d-45ef-876f-6c2e68859b9a 2026-02-24T18:16:22.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:16:22.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:16:22.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:16:22.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:330: TEST_divergent_ec: for i in $non_divergent 2026-02-24T18:16:22.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:332: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.2 2026-02-24T18:16:22.539 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:16:22.540 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:16:22.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:16:22.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:16:22.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:16:22.648 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:16:22.648 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:333: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 2 --op log --pgid 1.0 2026-02-24T18:16:22.648 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-24T18:16:22.648 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T18:16:22.648 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-24T18:16:22.648 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T18:16:22.649 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/divergent-priors/2 2026-02-24T18:16:22.649 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/2 --op log --pgid 1.0 2026-02-24T18:16:22.827 INFO:tasks.workunit.client.0.trial172.stderr:PG '1.0' not found 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1256: _objectstore_tool_nodown: return 1 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:334: TEST_divergent_ec: activate_osd td/divergent-priors 2 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/2 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:16:22.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:16:22.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:16:22.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:16:22.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/2 2026-02-24T18:16:22.940 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:16:22.941 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T18:16:22.941 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:16:22.941 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/2/whoami 2026-02-24T18:16:22.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T18:16:22.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:16:22.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:16:22.943 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:16:22.967 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:16:22.961+0000 7f2138641a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:16:22.969 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:16:22.961+0000 7f2138641a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:16:22.970 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:16:22.961+0000 7f2138641a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:16:23.159 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T18:16:23.159 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:16:23.160 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:16:23.160 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:16:23.160 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:16:23.160 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:16:23.160 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:16:23.160 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:16:23.161 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:16:23.161 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:16:23.376 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:16:23.369+0000 7ff9feab2a40 -1 osd.0 42 log_to_monitors true 2026-02-24T18:16:23.379 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 39 up_thru 0 down_at 33 last_clean_interval [15,29) [v2:127.0.0.1:6810/560661905,v1:127.0.0.1:6811/560661905] [v2:127.0.0.1:6812/560661905,v1:127.0.0.1:6813/560661905] exists,up 4e270c3f-1799-484f-8fb2-c5e0604b5c26 2026-02-24T18:16:23.380 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:16:23.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:16:23.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:16:23.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:336: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 1 --op log --pgid 1.0 2026-02-24T18:16:23.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-24T18:16:23.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T18:16:23.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-24T18:16:23.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T18:16:23.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-24T18:16:23.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op log --pgid 1.0 2026-02-24T18:16:23.464 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:16:23.457+0000 7f2138641a40 -1 Falling back to public interface 2026-02-24T18:16:23.637 INFO:tasks.workunit.client.0.trial172.stderr:PG '1.0' not found 2026-02-24T18:16:23.756 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1256: _objectstore_tool_nodown: return 1 2026-02-24T18:16:23.756 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:338: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-24T18:16:23.756 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:338: TEST_divergent_ec: wait_for_clean 2026-02-24T18:16:23.756 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:16:23.756 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:16:23.756 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:16:23.757 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-24T18:16:23.757 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:16:23.758 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:16:23.758 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:16:23.758 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:16:23.758 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:16:23.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-24T18:16:23.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:16:23.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:16:23.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:16:23.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:16:23.819 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:16:24.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:16:24.035 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:16:24.035 INFO:tasks.workunit.client.0.trial172.stderr:2' 2026-02-24T18:16:24.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:16:24.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:16:24.036 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:16:24.116 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=193273528323 2026-02-24T18:16:24.116 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 193273528323 2026-02-24T18:16:24.117 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-193273528323' 2026-02-24T18:16:24.117 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:16:24.117 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:16:24.182 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq= 2026-02-24T18:16:24.182 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: true 2026-02-24T18:16:24.182 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z '' 2026-02-24T18:16:24.182 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: flush_pg_stats: continue 2026-02-24T18:16:24.182 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:16:24.183 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:16:24.405 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:16:24.397+0000 7f2138641a40 -1 osd.2 42 log_to_monitors true 2026-02-24T18:16:29.271 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=210453397507 2026-02-24T18:16:29.271 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 210453397507 2026-02-24T18:16:29.271 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-193273528323 2-210453397507' 2026-02-24T18:16:29.271 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:16:29.271 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-193273528323 2026-02-24T18:16:29.271 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:16:29.273 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:16:29.273 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-193273528323 2026-02-24T18:16:29.274 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:16:29.275 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=193273528323 2026-02-24T18:16:29.275 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 193273528323' 2026-02-24T18:16:29.275 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 193273528323 2026-02-24T18:16:29.275 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:16:29.495 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 193273528324 -lt 193273528323 2026-02-24T18:16:29.495 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:16:29.496 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-210453397507 2026-02-24T18:16:29.496 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:16:29.497 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:16:29.497 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-210453397507 2026-02-24T18:16:29.498 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:16:29.498 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=210453397507 2026-02-24T18:16:29.499 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 210453397507 2026-02-24T18:16:29.500 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 210453397507' 2026-02-24T18:16:29.500 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:16:29.725 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 210453397507 -lt 210453397507 2026-02-24T18:16:29.725 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:16:29.726 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:16:29.726 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:16:30.013 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:16:30.013 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:16:30.014 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:16:30.014 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:16:30.014 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:16:30.014 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:16:30.014 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:16:30.014 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:16:30.226 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:16:30.226 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:16:30.227 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:16:30.227 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:16:30.507 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:16:30.507 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-24T18:16:30.507 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-24T18:16:30.507 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-24T18:16:30.507 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:16:30.507 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-24T18:16:30.608 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:16:30.609 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:16:30.610 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:16:30.610 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:16:30.610 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:16:30.610 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:16:30.610 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:16:30.610 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:16:30.828 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:16:30.829 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:16:30.829 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:16:30.829 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:16:31.116 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:16:31.116 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:16:31.116 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:16:31.116 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:16:31.116 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:16:31.116 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:16:31.116 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:16:31.116 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:16:31.117 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:16:31.396 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:16:31.397 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:16:31.397 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 8 )) 2026-02-24T18:16:31.397 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:16:31.397 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-24T18:16:31.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:16:31.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:16:31.598 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:16:31.598 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:16:31.598 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:16:31.598 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:16:31.598 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:16:31.598 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:16:31.815 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:16:31.816 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:16:31.816 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:16:31.816 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:16:32.096 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:16:32.096 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:16:32.096 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:16:32.096 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:16:32.096 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:16:32.096 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:16:32.096 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:16:32.097 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:16:32.097 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:16:32.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:16:32.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:16:32.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 8 )) 2026-02-24T18:16:32.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:16:32.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-24T18:16:32.781 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:16:32.781 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:16:32.782 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:16:32.782 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:16:32.782 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:16:32.782 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:16:32.782 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:16:32.782 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:16:33.005 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:16:33.005 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:16:33.005 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:16:33.005 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:16:33.297 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:16:33.297 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:16:33.297 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:16:33.297 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:16:33.298 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:16:33.298 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:16:33.298 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:16:33.298 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:16:33.298 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:16:33.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:16:33.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:16:33.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 8 )) 2026-02-24T18:16:33.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:16:33.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-24T18:16:34.393 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:16:34.393 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:16:34.393 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:16:34.393 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:16:34.393 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:16:34.393 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:16:34.393 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:16:34.393 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:16:34.611 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:16:34.612 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:16:34.612 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:16:34.612 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:16:34.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:16:34.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:16:34.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:16:34.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:16:34.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:16:34.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:16:34.897 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:16:34.897 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:16:34.897 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:16:35.182 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:16:35.182 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:16:35.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 8 )) 2026-02-24T18:16:35.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:16:35.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 1.6 2026-02-24T18:16:36.783 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:16:36.783 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:16:36.783 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:16:36.784 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:16:36.784 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:16:36.784 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:16:36.784 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:16:36.784 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:16:36.994 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:16:36.995 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:16:36.995 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:16:36.995 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:16:37.273 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:16:37.273 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:16:37.273 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:16:37.273 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:16:37.273 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:16:37.273 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:16:37.273 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:16:37.274 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:16:37.274 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:16:37.568 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:16:37.568 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:16:37.568 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 5 >= 8 )) 2026-02-24T18:16:37.568 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:16:37.569 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 3.2 2026-02-24T18:16:40.769 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:16:40.769 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:16:40.770 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:16:40.770 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:16:40.771 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:16:40.771 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:16:40.771 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:16:40.771 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:16:40.986 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:16:40.986 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:16:40.987 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:16:40.987 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:16:41.275 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:16:41.275 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:16:41.275 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:16:41.276 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:16:41.276 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:16:41.276 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:16:41.276 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:16:41.276 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:16:41.276 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:16:41.570 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:16:41.571 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:16:41.571 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 6 >= 8 )) 2026-02-24T18:16:41.571 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:16:41.571 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 6.4 2026-02-24T18:16:47.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:16:47.972 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:16:47.972 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:16:47.972 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:16:47.972 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:16:47.972 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:16:47.972 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:16:47.973 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:16:48.194 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:16:48.195 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:16:48.195 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:16:48.195 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:16:48.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:16:48.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:16:48.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:16:48.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:16:48.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:16:48.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:16:48.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:16:48.486 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:16:48.486 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:16:48.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:16:48.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:16:48.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 7 >= 8 )) 2026-02-24T18:16:48.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T18:16:48.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 7.3 2026-02-24T18:16:56.077 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T18:16:56.077 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:16:56.077 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:16:56.077 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:16:56.078 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:16:56.078 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:16:56.079 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:16:56.079 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:16:56.296 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-24T18:16:56.297 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:16:56.297 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:16:56.298 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:16:56.588 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-24T18:16:56.588 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-24T18:16:56.588 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T18:16:56.588 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T18:16:56.589 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T18:16:56.589 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T18:16:56.589 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T18:16:56.589 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T18:16:56.589 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T18:16:56.879 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T18:16:56.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T18:16:56.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 8 >= 8 )) 2026-02-24T18:16:56.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-24T18:16:57.168 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:16:57.168 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_fingerprint": "70393c9b-3fd2-4a6c-9859-cb6a60af12dc", 2026-02-24T18:16:57.168 INFO:tasks.workunit.client.0.trial172.stdout: "version": "20.3.0-5507-g618c318a", 2026-02-24T18:16:57.168 INFO:tasks.workunit.client.0.trial172.stdout: "commit": "618c318a7a0b287ab5aeff207b7bf1d73ef45044", 2026-02-24T18:16:57.168 INFO:tasks.workunit.client.0.trial172.stdout: "timestamp": "2026-02-24T18:16:57.165093+0000", 2026-02-24T18:16:57.168 INFO:tasks.workunit.client.0.trial172.stdout: "tag": "", 2026-02-24T18:16:57.168 INFO:tasks.workunit.client.0.trial172.stdout: "health": { 2026-02-24T18:16:57.169 INFO:tasks.workunit.client.0.trial172.stdout: "status": "HEALTH_WARN", 2026-02-24T18:16:57.169 INFO:tasks.workunit.client.0.trial172.stdout: "checks": { 2026-02-24T18:16:57.169 INFO:tasks.workunit.client.0.trial172.stdout: "OSDMAP_FLAGS": { 2026-02-24T18:16:57.169 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:16:57.169 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:16:57.169 INFO:tasks.workunit.client.0.trial172.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-24T18:16:57.169 INFO:tasks.workunit.client.0.trial172.stdout: "count": 17 2026-02-24T18:16:57.169 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.169 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [], 2026-02-24T18:16:57.169 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:16:57.169 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.169 INFO:tasks.workunit.client.0.trial172.stdout: "OSD_DOWN": { 2026-02-24T18:16:57.170 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:16:57.170 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:16:57.170 INFO:tasks.workunit.client.0.trial172.stdout: "message": "1 osds down", 2026-02-24T18:16:57.170 INFO:tasks.workunit.client.0.trial172.stdout: "count": 1 2026-02-24T18:16:57.170 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.170 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [ 2026-02-24T18:16:57.170 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.170 INFO:tasks.workunit.client.0.trial172.stdout: "message": "osd.1 (root=default,host=trial172) is down" 2026-02-24T18:16:57.170 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.170 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.170 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:16:57.170 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.171 INFO:tasks.workunit.client.0.trial172.stdout: "PG_DEGRADED": { 2026-02-24T18:16:57.171 INFO:tasks.workunit.client.0.trial172.stdout: "severity": "HEALTH_WARN", 2026-02-24T18:16:57.171 INFO:tasks.workunit.client.0.trial172.stdout: "summary": { 2026-02-24T18:16:57.171 INFO:tasks.workunit.client.0.trial172.stdout: "message": "Degraded data redundancy: 101/303 objects degraded (33.333%), 1 pg degraded", 2026-02-24T18:16:57.171 INFO:tasks.workunit.client.0.trial172.stdout: "count": 1 2026-02-24T18:16:57.171 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.171 INFO:tasks.workunit.client.0.trial172.stdout: "detail": [ 2026-02-24T18:16:57.171 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.171 INFO:tasks.workunit.client.0.trial172.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [NONE,0,2]" 2026-02-24T18:16:57.171 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.171 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.171 INFO:tasks.workunit.client.0.trial172.stdout: "muted": false 2026-02-24T18:16:57.172 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.172 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.172 INFO:tasks.workunit.client.0.trial172.stdout: "mutes": [] 2026-02-24T18:16:57.172 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.172 INFO:tasks.workunit.client.0.trial172.stdout: "monmap_first_committed": 1, 2026-02-24T18:16:57.172 INFO:tasks.workunit.client.0.trial172.stdout: "monmap_last_committed": 1, 2026-02-24T18:16:57.172 INFO:tasks.workunit.client.0.trial172.stdout: "monmap": { 2026-02-24T18:16:57.172 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 1, 2026-02-24T18:16:57.172 INFO:tasks.workunit.client.0.trial172.stdout: "fsid": "27512777-e37f-4f16-b176-1621842bd4df", 2026-02-24T18:16:57.172 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:14:58.797613Z", 2026-02-24T18:16:57.172 INFO:tasks.workunit.client.0.trial172.stdout: "created": "2026-02-24T18:14:58.797613Z", 2026-02-24T18:16:57.172 INFO:tasks.workunit.client.0.trial172.stdout: "min_mon_release": 20, 2026-02-24T18:16:57.173 INFO:tasks.workunit.client.0.trial172.stdout: "min_mon_release_name": "tentacle", 2026-02-24T18:16:57.173 INFO:tasks.workunit.client.0.trial172.stdout: "election_strategy": 1, 2026-02-24T18:16:57.173 INFO:tasks.workunit.client.0.trial172.stdout: "disallowed_leaders": "", 2026-02-24T18:16:57.173 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode": false, 2026-02-24T18:16:57.173 INFO:tasks.workunit.client.0.trial172.stdout: "tiebreaker_mon": "", 2026-02-24T18:16:57.173 INFO:tasks.workunit.client.0.trial172.stdout: "removed_ranks": "", 2026-02-24T18:16:57.173 INFO:tasks.workunit.client.0.trial172.stdout: "features": { 2026-02-24T18:16:57.173 INFO:tasks.workunit.client.0.trial172.stdout: "persistent": [ 2026-02-24T18:16:57.173 INFO:tasks.workunit.client.0.trial172.stdout: "kraken", 2026-02-24T18:16:57.173 INFO:tasks.workunit.client.0.trial172.stdout: "luminous", 2026-02-24T18:16:57.173 INFO:tasks.workunit.client.0.trial172.stdout: "mimic", 2026-02-24T18:16:57.173 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap-prune", 2026-02-24T18:16:57.173 INFO:tasks.workunit.client.0.trial172.stdout: "nautilus", 2026-02-24T18:16:57.174 INFO:tasks.workunit.client.0.trial172.stdout: "octopus", 2026-02-24T18:16:57.174 INFO:tasks.workunit.client.0.trial172.stdout: "pacific", 2026-02-24T18:16:57.174 INFO:tasks.workunit.client.0.trial172.stdout: "elector-pinging", 2026-02-24T18:16:57.174 INFO:tasks.workunit.client.0.trial172.stdout: "quincy", 2026-02-24T18:16:57.174 INFO:tasks.workunit.client.0.trial172.stdout: "reef", 2026-02-24T18:16:57.174 INFO:tasks.workunit.client.0.trial172.stdout: "squid", 2026-02-24T18:16:57.174 INFO:tasks.workunit.client.0.trial172.stdout: "tentacle", 2026-02-24T18:16:57.174 INFO:tasks.workunit.client.0.trial172.stdout: "nvmeof_beacon_diff" 2026-02-24T18:16:57.174 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.174 INFO:tasks.workunit.client.0.trial172.stdout: "optional": [] 2026-02-24T18:16:57.174 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.174 INFO:tasks.workunit.client.0.trial172.stdout: "mons": [ 2026-02-24T18:16:57.174 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.175 INFO:tasks.workunit.client.0.trial172.stdout: "rank": 0, 2026-02-24T18:16:57.175 INFO:tasks.workunit.client.0.trial172.stdout: "name": "a", 2026-02-24T18:16:57.175 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:16:57.175 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:57.175 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.175 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:57.175 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:7115", 2026-02-24T18:16:57.175 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 0 2026-02-24T18:16:57.175 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.175 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.175 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.175 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:7115/0", 2026-02-24T18:16:57.176 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-24T18:16:57.176 INFO:tasks.workunit.client.0.trial172.stdout: "priority": 0, 2026-02-24T18:16:57.176 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 0, 2026-02-24T18:16:57.176 INFO:tasks.workunit.client.0.trial172.stdout: "time_added": "2026-02-24T18:14:58:797607+0000", 2026-02-24T18:16:57.176 INFO:tasks.workunit.client.0.trial172.stdout: "crush_location": [] 2026-02-24T18:16:57.176 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.176 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.176 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.176 INFO:tasks.workunit.client.0.trial172.stdout: "quorum": [ 2026-02-24T18:16:57.176 INFO:tasks.workunit.client.0.trial172.stdout: 0 2026-02-24T18:16:57.176 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.176 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap": { 2026-02-24T18:16:57.176 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 50, 2026-02-24T18:16:57.177 INFO:tasks.workunit.client.0.trial172.stdout: "fsid": "27512777-e37f-4f16-b176-1621842bd4df", 2026-02-24T18:16:57.177 INFO:tasks.workunit.client.0.trial172.stdout: "created": "2026-02-24T18:14:58.882667+0000", 2026-02-24T18:16:57.177 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:16:24.761609+0000", 2026-02-24T18:16:57.177 INFO:tasks.workunit.client.0.trial172.stdout: "last_up_change": "2026-02-24T18:16:24.659027+0000", 2026-02-24T18:16:57.177 INFO:tasks.workunit.client.0.trial172.stdout: "last_in_change": "2026-02-24T18:15:46.707383+0000", 2026-02-24T18:16:57.177 INFO:tasks.workunit.client.0.trial172.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-24T18:16:57.177 INFO:tasks.workunit.client.0.trial172.stdout: "flags_num": 5800384, 2026-02-24T18:16:57.177 INFO:tasks.workunit.client.0.trial172.stdout: "flags_set": [ 2026-02-24T18:16:57.177 INFO:tasks.workunit.client.0.trial172.stdout: "nodown", 2026-02-24T18:16:57.177 INFO:tasks.workunit.client.0.trial172.stdout: "noin", 2026-02-24T18:16:57.177 INFO:tasks.workunit.client.0.trial172.stdout: "noout", 2026-02-24T18:16:57.178 INFO:tasks.workunit.client.0.trial172.stdout: "pglog_hardlimit", 2026-02-24T18:16:57.178 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snapdirs", 2026-02-24T18:16:57.178 INFO:tasks.workunit.client.0.trial172.stdout: "recovery_deletes", 2026-02-24T18:16:57.178 INFO:tasks.workunit.client.0.trial172.stdout: "sortbitwise" 2026-02-24T18:16:57.178 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.178 INFO:tasks.workunit.client.0.trial172.stdout: "crush_version": 12, 2026-02-24T18:16:57.178 INFO:tasks.workunit.client.0.trial172.stdout: "full_ratio": 0.99000000953674316, 2026-02-24T18:16:57.178 INFO:tasks.workunit.client.0.trial172.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-24T18:16:57.178 INFO:tasks.workunit.client.0.trial172.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-24T18:16:57.178 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_snapshot": "", 2026-02-24T18:16:57.178 INFO:tasks.workunit.client.0.trial172.stdout: "pool_max": 1, 2026-02-24T18:16:57.178 INFO:tasks.workunit.client.0.trial172.stdout: "max_osd": 3, 2026-02-24T18:16:57.178 INFO:tasks.workunit.client.0.trial172.stdout: "require_min_compat_client": "luminous", 2026-02-24T18:16:57.179 INFO:tasks.workunit.client.0.trial172.stdout: "min_compat_client": "jewel", 2026-02-24T18:16:57.179 INFO:tasks.workunit.client.0.trial172.stdout: "require_osd_release": "tentacle", 2026-02-24T18:16:57.179 INFO:tasks.workunit.client.0.trial172.stdout: "allow_crimson": false, 2026-02-24T18:16:57.179 INFO:tasks.workunit.client.0.trial172.stdout: "pools": [ 2026-02-24T18:16:57.179 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.179 INFO:tasks.workunit.client.0.trial172.stdout: "pool": 1, 2026-02-24T18:16:57.179 INFO:tasks.workunit.client.0.trial172.stdout: "pool_name": "test", 2026-02-24T18:16:57.179 INFO:tasks.workunit.client.0.trial172.stdout: "create_time": "2026-02-24T18:15:12.294587+0000", 2026-02-24T18:16:57.179 INFO:tasks.workunit.client.0.trial172.stdout: "flags": 16389, 2026-02-24T18:16:57.179 INFO:tasks.workunit.client.0.trial172.stdout: "flags_names": "hashpspool,ec_overwrites,pool_snaps", 2026-02-24T18:16:57.179 INFO:tasks.workunit.client.0.trial172.stdout: "type": 3, 2026-02-24T18:16:57.179 INFO:tasks.workunit.client.0.trial172.stdout: "size": 3, 2026-02-24T18:16:57.179 INFO:tasks.workunit.client.0.trial172.stdout: "min_size": 2, 2026-02-24T18:16:57.179 INFO:tasks.workunit.client.0.trial172.stdout: "crush_rule": 1, 2026-02-24T18:16:57.180 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_count": 0, 2026-02-24T18:16:57.180 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_target": 0, 2026-02-24T18:16:57.180 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_barrier": 0, 2026-02-24T18:16:57.180 INFO:tasks.workunit.client.0.trial172.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-24T18:16:57.180 INFO:tasks.workunit.client.0.trial172.stdout: "is_stretch_pool": false, 2026-02-24T18:16:57.180 INFO:tasks.workunit.client.0.trial172.stdout: "object_hash": 2, 2026-02-24T18:16:57.180 INFO:tasks.workunit.client.0.trial172.stdout: "pg_autoscale_mode": "off", 2026-02-24T18:16:57.180 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num": 1, 2026-02-24T18:16:57.180 INFO:tasks.workunit.client.0.trial172.stdout: "pg_placement_num": 1, 2026-02-24T18:16:57.180 INFO:tasks.workunit.client.0.trial172.stdout: "pg_placement_num_target": 1, 2026-02-24T18:16:57.180 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num_target": 1, 2026-02-24T18:16:57.180 INFO:tasks.workunit.client.0.trial172.stdout: "pg_num_pending": 1, 2026-02-24T18:16:57.181 INFO:tasks.workunit.client.0.trial172.stdout: "last_pg_merge_meta": { 2026-02-24T18:16:57.181 INFO:tasks.workunit.client.0.trial172.stdout: "source_pgid": "0.0", 2026-02-24T18:16:57.181 INFO:tasks.workunit.client.0.trial172.stdout: "ready_epoch": 0, 2026-02-24T18:16:57.181 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_started": 0, 2026-02-24T18:16:57.181 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_clean": 0, 2026-02-24T18:16:57.181 INFO:tasks.workunit.client.0.trial172.stdout: "source_version": "0'0", 2026-02-24T18:16:57.181 INFO:tasks.workunit.client.0.trial172.stdout: "target_version": "0'0" 2026-02-24T18:16:57.181 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.181 INFO:tasks.workunit.client.0.trial172.stdout: "last_change": "30", 2026-02-24T18:16:57.181 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend": "0", 2026-02-24T18:16:57.181 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-24T18:16:57.181 INFO:tasks.workunit.client.0.trial172.stdout: "last_force_op_resend_preluminous": "0", 2026-02-24T18:16:57.181 INFO:tasks.workunit.client.0.trial172.stdout: "auid": 0, 2026-02-24T18:16:57.182 INFO:tasks.workunit.client.0.trial172.stdout: "snap_mode": "pool", 2026-02-24T18:16:57.182 INFO:tasks.workunit.client.0.trial172.stdout: "snap_seq": 1, 2026-02-24T18:16:57.182 INFO:tasks.workunit.client.0.trial172.stdout: "snap_epoch": 30, 2026-02-24T18:16:57.182 INFO:tasks.workunit.client.0.trial172.stdout: "pool_snaps": [ 2026-02-24T18:16:57.182 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.182 INFO:tasks.workunit.client.0.trial172.stdout: "snapid": 1, 2026-02-24T18:16:57.182 INFO:tasks.workunit.client.0.trial172.stdout: "stamp": "2026-02-24T18:15:30.759768+0000", 2026-02-24T18:16:57.182 INFO:tasks.workunit.client.0.trial172.stdout: "name": "snap1" 2026-02-24T18:16:57.182 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.182 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.182 INFO:tasks.workunit.client.0.trial172.stdout: "removed_snaps": "[]", 2026-02-24T18:16:57.182 INFO:tasks.workunit.client.0.trial172.stdout: "quota_max_bytes": 0, 2026-02-24T18:16:57.183 INFO:tasks.workunit.client.0.trial172.stdout: "quota_max_objects": 0, 2026-02-24T18:16:57.183 INFO:tasks.workunit.client.0.trial172.stdout: "tiers": [], 2026-02-24T18:16:57.183 INFO:tasks.workunit.client.0.trial172.stdout: "tier_of": -1, 2026-02-24T18:16:57.183 INFO:tasks.workunit.client.0.trial172.stdout: "read_tier": -1, 2026-02-24T18:16:57.183 INFO:tasks.workunit.client.0.trial172.stdout: "write_tier": -1, 2026-02-24T18:16:57.183 INFO:tasks.workunit.client.0.trial172.stdout: "cache_mode": "none", 2026-02-24T18:16:57.183 INFO:tasks.workunit.client.0.trial172.stdout: "target_max_bytes": 0, 2026-02-24T18:16:57.183 INFO:tasks.workunit.client.0.trial172.stdout: "target_max_objects": 0, 2026-02-24T18:16:57.183 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-24T18:16:57.183 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-24T18:16:57.183 INFO:tasks.workunit.client.0.trial172.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-24T18:16:57.183 INFO:tasks.workunit.client.0.trial172.stdout: "cache_min_flush_age": 0, 2026-02-24T18:16:57.183 INFO:tasks.workunit.client.0.trial172.stdout: "cache_min_evict_age": 0, 2026-02-24T18:16:57.184 INFO:tasks.workunit.client.0.trial172.stdout: "erasure_code_profile": "myprofile", 2026-02-24T18:16:57.184 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_params": { 2026-02-24T18:16:57.184 INFO:tasks.workunit.client.0.trial172.stdout: "type": "none" 2026-02-24T18:16:57.184 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.184 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_period": 0, 2026-02-24T18:16:57.184 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_count": 0, 2026-02-24T18:16:57.184 INFO:tasks.workunit.client.0.trial172.stdout: "use_gmt_hitset": true, 2026-02-24T18:16:57.184 INFO:tasks.workunit.client.0.trial172.stdout: "min_read_recency_for_promote": 0, 2026-02-24T18:16:57.184 INFO:tasks.workunit.client.0.trial172.stdout: "min_write_recency_for_promote": 0, 2026-02-24T18:16:57.184 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_grade_decay_rate": 0, 2026-02-24T18:16:57.184 INFO:tasks.workunit.client.0.trial172.stdout: "hit_set_search_last_n": 0, 2026-02-24T18:16:57.184 INFO:tasks.workunit.client.0.trial172.stdout: "grade_table": [], 2026-02-24T18:16:57.184 INFO:tasks.workunit.client.0.trial172.stdout: "stripe_width": 8192, 2026-02-24T18:16:57.184 INFO:tasks.workunit.client.0.trial172.stdout: "expected_num_objects": 0, 2026-02-24T18:16:57.185 INFO:tasks.workunit.client.0.trial172.stdout: "fast_read": false, 2026-02-24T18:16:57.185 INFO:tasks.workunit.client.0.trial172.stdout: "nonprimary_shards": "{}", 2026-02-24T18:16:57.185 INFO:tasks.workunit.client.0.trial172.stdout: "options": {}, 2026-02-24T18:16:57.185 INFO:tasks.workunit.client.0.trial172.stdout: "application_metadata": {} 2026-02-24T18:16:57.185 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.185 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.185 INFO:tasks.workunit.client.0.trial172.stdout: "osds": [ 2026-02-24T18:16:57.185 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.185 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:16:57.185 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "93b13c7c-e44d-45ef-876f-6c2e68859b9a", 2026-02-24T18:16:57.185 INFO:tasks.workunit.client.0.trial172.stdout: "up": 1, 2026-02-24T18:16:57.185 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:16:57.185 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:16:57.186 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:16:57.186 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 37, 2026-02-24T18:16:57.186 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 42, 2026-02-24T18:16:57.186 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 45, 2026-02-24T18:16:57.186 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 49, 2026-02-24T18:16:57.186 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 44, 2026-02-24T18:16:57.186 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:16:57.186 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:16:57.186 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:57.186 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.186 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:57.186 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6800", 2026-02-24T18:16:57.187 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2122755957 2026-02-24T18:16:57.187 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.187 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.187 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:57.187 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6801", 2026-02-24T18:16:57.187 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2122755957 2026-02-24T18:16:57.187 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.187 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.187 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.187 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:16:57.187 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:57.187 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.187 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:57.188 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6802", 2026-02-24T18:16:57.188 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2122755957 2026-02-24T18:16:57.188 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.188 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.188 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:57.188 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6803", 2026-02-24T18:16:57.188 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2122755957 2026-02-24T18:16:57.188 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.188 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.188 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.188 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:16:57.188 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:57.188 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.189 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:57.189 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6806", 2026-02-24T18:16:57.189 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2122755957 2026-02-24T18:16:57.189 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.189 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.189 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:57.189 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6807", 2026-02-24T18:16:57.189 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2122755957 2026-02-24T18:16:57.189 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.189 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.189 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.189 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:16:57.190 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:57.190 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.190 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:57.190 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6804", 2026-02-24T18:16:57.190 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2122755957 2026-02-24T18:16:57.190 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.190 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.190 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:57.190 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6805", 2026-02-24T18:16:57.190 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 2122755957 2026-02-24T18:16:57.190 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.190 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.191 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.191 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6801/2122755957", 2026-02-24T18:16:57.191 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6803/2122755957", 2026-02-24T18:16:57.191 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6807/2122755957", 2026-02-24T18:16:57.191 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6805/2122755957", 2026-02-24T18:16:57.191 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:16:57.191 INFO:tasks.workunit.client.0.trial172.stdout: "exists", 2026-02-24T18:16:57.191 INFO:tasks.workunit.client.0.trial172.stdout: "up" 2026-02-24T18:16:57.191 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.191 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.191 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.192 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:16:57.192 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "a1643499-a87c-4291-842c-c57767e846a8", 2026-02-24T18:16:57.192 INFO:tasks.workunit.client.0.trial172.stdout: "up": 0, 2026-02-24T18:16:57.192 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:16:57.192 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:16:57.192 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:16:57.192 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 0, 2026-02-24T18:16:57.192 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 0, 2026-02-24T18:16:57.192 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 10, 2026-02-24T18:16:57.193 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 25, 2026-02-24T18:16:57.193 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 32, 2026-02-24T18:16:57.193 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:16:57.193 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:16:57.193 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:57.193 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.193 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:57.193 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6810", 2026-02-24T18:16:57.193 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:57.193 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.194 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.194 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:57.194 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6811", 2026-02-24T18:16:57.194 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:57.194 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.194 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.194 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.194 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:16:57.194 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:57.194 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.194 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:57.195 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6812", 2026-02-24T18:16:57.195 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:57.195 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.195 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.195 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:57.195 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6813", 2026-02-24T18:16:57.195 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:57.195 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.195 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.195 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.195 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:16:57.196 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:57.196 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.196 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:57.196 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6816", 2026-02-24T18:16:57.196 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:57.196 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.196 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.196 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:57.196 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6817", 2026-02-24T18:16:57.196 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:57.197 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.197 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.197 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.197 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:16:57.197 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:57.197 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.197 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:57.197 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6814", 2026-02-24T18:16:57.197 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:57.197 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.197 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.198 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:57.198 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6815", 2026-02-24T18:16:57.198 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 427574815 2026-02-24T18:16:57.198 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.198 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.198 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.198 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6811/427574815", 2026-02-24T18:16:57.198 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6813/427574815", 2026-02-24T18:16:57.198 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6817/427574815", 2026-02-24T18:16:57.198 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6815/427574815", 2026-02-24T18:16:57.198 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:16:57.199 INFO:tasks.workunit.client.0.trial172.stdout: "exists" 2026-02-24T18:16:57.199 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.199 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.199 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.199 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:16:57.199 INFO:tasks.workunit.client.0.trial172.stdout: "uuid": "4e270c3f-1799-484f-8fb2-c5e0604b5c26", 2026-02-24T18:16:57.199 INFO:tasks.workunit.client.0.trial172.stdout: "up": 1, 2026-02-24T18:16:57.199 INFO:tasks.workunit.client.0.trial172.stdout: "in": 1, 2026-02-24T18:16:57.199 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 1, 2026-02-24T18:16:57.199 INFO:tasks.workunit.client.0.trial172.stdout: "primary_affinity": 1, 2026-02-24T18:16:57.199 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_begin": 39, 2026-02-24T18:16:57.200 INFO:tasks.workunit.client.0.trial172.stdout: "last_clean_end": 42, 2026-02-24T18:16:57.200 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 49, 2026-02-24T18:16:57.200 INFO:tasks.workunit.client.0.trial172.stdout: "up_thru": 0, 2026-02-24T18:16:57.200 INFO:tasks.workunit.client.0.trial172.stdout: "down_at": 48, 2026-02-24T18:16:57.200 INFO:tasks.workunit.client.0.trial172.stdout: "lost_at": 0, 2026-02-24T18:16:57.200 INFO:tasks.workunit.client.0.trial172.stdout: "public_addrs": { 2026-02-24T18:16:57.200 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:57.200 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.200 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:57.200 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6810", 2026-02-24T18:16:57.200 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3138611225 2026-02-24T18:16:57.200 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.200 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.201 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:57.201 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6811", 2026-02-24T18:16:57.201 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3138611225 2026-02-24T18:16:57.201 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.201 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.201 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.201 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addrs": { 2026-02-24T18:16:57.201 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:57.201 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.201 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:57.201 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6812", 2026-02-24T18:16:57.201 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3138611225 2026-02-24T18:16:57.201 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.202 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.202 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:57.202 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6813", 2026-02-24T18:16:57.202 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3138611225 2026-02-24T18:16:57.202 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.202 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.202 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.202 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addrs": { 2026-02-24T18:16:57.202 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:57.202 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.202 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:57.202 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6816", 2026-02-24T18:16:57.203 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3138611225 2026-02-24T18:16:57.203 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.203 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.203 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:57.203 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6817", 2026-02-24T18:16:57.203 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3138611225 2026-02-24T18:16:57.203 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.203 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.203 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.203 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addrs": { 2026-02-24T18:16:57.203 INFO:tasks.workunit.client.0.trial172.stdout: "addrvec": [ 2026-02-24T18:16:57.203 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.203 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v2", 2026-02-24T18:16:57.203 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6814", 2026-02-24T18:16:57.204 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3138611225 2026-02-24T18:16:57.204 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.204 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.204 INFO:tasks.workunit.client.0.trial172.stdout: "type": "v1", 2026-02-24T18:16:57.204 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "127.0.0.1:6815", 2026-02-24T18:16:57.204 INFO:tasks.workunit.client.0.trial172.stdout: "nonce": 3138611225 2026-02-24T18:16:57.204 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.204 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.204 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.204 INFO:tasks.workunit.client.0.trial172.stdout: "public_addr": "127.0.0.1:6811/3138611225", 2026-02-24T18:16:57.204 INFO:tasks.workunit.client.0.trial172.stdout: "cluster_addr": "127.0.0.1:6813/3138611225", 2026-02-24T18:16:57.204 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3138611225", 2026-02-24T18:16:57.204 INFO:tasks.workunit.client.0.trial172.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3138611225", 2026-02-24T18:16:57.204 INFO:tasks.workunit.client.0.trial172.stdout: "state": [ 2026-02-24T18:16:57.205 INFO:tasks.workunit.client.0.trial172.stdout: "exists", 2026-02-24T18:16:57.205 INFO:tasks.workunit.client.0.trial172.stdout: "up" 2026-02-24T18:16:57.205 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.205 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.205 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.205 INFO:tasks.workunit.client.0.trial172.stdout: "osd_xinfo": [ 2026-02-24T18:16:57.205 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.205 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:16:57.205 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:16:23.550728+0000", 2026-02-24T18:16:57.205 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:16:57.205 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:16:57.205 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:16:57.205 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:16:57.205 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:15:02.261828+0000", 2026-02-24T18:16:57.205 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:16:57.206 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.206 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.206 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:16:57.206 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:15:41.701782+0000", 2026-02-24T18:16:57.206 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:16:57.206 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:16:57.206 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:16:57.206 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:16:57.206 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:15:06.087865+0000", 2026-02-24T18:16:57.206 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:16:57.206 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.206 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.206 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: "down_stamp": "2026-02-24T18:16:24.558347+0000", 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_probability": 0, 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: "laggy_interval": 0, 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: "features": 4544132024016699391, 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: "old_weight": 0, 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: "last_purged_snaps_scrub": "2026-02-24T18:15:10.037025+0000", 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: "dead_epoch": 0 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap": [], 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap_items": [], 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: "pg_upmap_primaries": [], 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: "pg_temp": [], 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: "primary_temp": [], 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: "blocklist": {}, 2026-02-24T18:16:57.207 INFO:tasks.workunit.client.0.trial172.stdout: "range_blocklist": {}, 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: "erasure_code_profiles": { 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: "default": { 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: "k": "2", 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: "m": "2", 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: "plugin": "isa", 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: "technique": "reed_sol_van" 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: "myprofile": { 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: "crush-device-class": "", 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: "crush-failure-domain": "osd", 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: "crush-num-failure-domains": "0", 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: "crush-osds-per-failure-domain": "0", 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: "crush-root": "default", 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: "k": "2", 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: "m": "1", 2026-02-24T18:16:57.208 INFO:tasks.workunit.client.0.trial172.stdout: "plugin": "isa", 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: "technique": "reed_sol_van" 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: "removed_snaps_queue": [], 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: "new_removed_snaps": [], 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: "new_purged_snaps": [], 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: "crush_node_flags": {}, 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: "device_class_flags": {}, 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode": { 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode_enabled": false, 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_bucket_count": 0, 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: "degraded_stretch_mode": 0, 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: "recovering_stretch_mode": 0, 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: "stretch_mode_bucket": 0 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.209 INFO:tasks.workunit.client.0.trial172.stdout: "osd_metadata": [ 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6802/2122755957,v1:127.0.0.1:6803/2122755957]", 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:16:57.210 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:14:59.857538Z", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:16:57.211 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6800/2122755957,v1:127.0.0.1:6801/2122755957]", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/2122755957,v1:127.0.0.1:6807/2122755957]", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/2122755957,v1:127.0.0.1:6805/2122755957]", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:16:57.212 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/0", 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6812/427574815,v1:127.0.0.1:6813/427574815]", 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:16:57.213 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:15:03.492854Z", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:16:57.214 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6810/427574815,v1:127.0.0.1:6811/427574815]", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/427574815,v1:127.0.0.1:6817/427574815]", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/427574815,v1:127.0.0.1:6815/427574815]", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/1", 2026-02-24T18:16:57.215 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "arch": "x86_64", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "back_addr": "[v2:127.0.0.1:6812/3138611225,v1:127.0.0.1:6813/3138611225]", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "back_iface": "", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs": "1", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_db": "0", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_dedicated_wal": "0", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "bluefs_single_shared_device": "1", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocation_from_file": "1", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_allocator": "hybrid", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_access_mode": "file", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_block_size": "4096", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-24T18:16:57.216 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_rotational": "0", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_size": "107374182400", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_support_discard": "1", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_bdev_type": "ssd", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_db_sharding": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_min_alloc_size": "4096", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "bluestore_write_mode": "classic", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_release": "tentacle", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_short": "20.3.0-5507-g618c318a", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "ceph_version_when_created": "ceph version 20.3.0-5507-g618c318a (618c318a7a0b287ab5aeff207b7bf1d73ef45044) tentacle (dev - None)", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "created_at": "2026-02-24T18:15:07.600471Z", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "default_device_class": "ssd", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6749", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "devices": "nvme0n1", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "distro": "ubuntu", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "distro_version": "22.04", 2026-02-24T18:16:57.217 INFO:tasks.workunit.client.0.trial172.stdout: "front_addr": "[v2:127.0.0.1:6810/3138611225,v1:127.0.0.1:6811/3138611225]", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "front_iface": "", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3138611225,v1:127.0.0.1:6817/3138611225]", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3138611225,v1:127.0.0.1:6815/3138611225]", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "hostname": "trial172", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "journal_rotational": "0", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_description": "#181-Ubuntu SMP Fri Feb 6 22:44:50 UTC 2026", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "kernel_version": "5.15.0-171-generic", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "mem_swap_kb": "8388604", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "mem_total_kb": "130961700", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_node": "0", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "objectstore_numa_nodes": "0", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "os": "Linux", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "osd_data": "td/divergent-priors/2", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "osd_objectstore": "bluestore", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "osdspec_affinity": "", 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "rotational": "0" 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_clean_epochs": { 2026-02-24T18:16:57.218 INFO:tasks.workunit.client.0.trial172.stdout: "min_last_epoch_clean": 26, 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: "last_epoch_clean": { 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: "per_pool": [ 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": 1, 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: "floor": 26 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: "osd_epochs": [ 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 46 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 50 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_first_committed": 1, 2026-02-24T18:16:57.219 INFO:tasks.workunit.client.0.trial172.stdout: "osdmap_last_committed": 50, 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: "crushmap": { 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: "devices": [ 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.0", 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.1", 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd.2", 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: "class": "ssd" 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: "types": [ 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.220 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 0, 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: "name": "osd" 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: "name": "host" 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 2, 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: "name": "chassis" 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 3, 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: "name": "rack" 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 4, 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: "name": "row" 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 5, 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: "name": "pdu" 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.221 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 6, 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: "name": "pod" 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 7, 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: "name": "room" 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 8, 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: "name": "datacenter" 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 9, 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: "name": "zone" 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 10, 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: "name": "region" 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: "name": "root" 2026-02-24T18:16:57.222 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "buckets": [ 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "id": -1, 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "name": "default", 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "root", 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "id": -3, 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "id": -2, 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "name": "default~ssd", 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 11, 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "root", 2026-02-24T18:16:57.223 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "id": -4, 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "id": -3, 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "name": "trial172", 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "host", 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:16:57.224 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 1 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 2 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "id": -4, 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "name": "trial172~ssd", 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "type_id": 1, 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "type_name": "host", 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 19206, 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "alg": "straw2", 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "hash": "rjenkins1", 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "items": [ 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "id": 0, 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:16:57.225 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 0 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: "id": 1, 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 1 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: "id": 2, 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: "weight": 6402, 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: "pos": 2 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: "rules": [ 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: "rule_id": 0, 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: "rule_name": "replicated_rule", 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: "type": 1, 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: "steps": [ 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: "op": "take", 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: "item": -1, 2026-02-24T18:16:57.226 INFO:tasks.workunit.client.0.trial172.stdout: "item_name": "default" 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: "op": "choose_firstn", 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: "num": 0, 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: "type": "osd" 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: "op": "emit" 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: "rule_id": 1, 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: "rule_name": "test", 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: "type": 3, 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: "steps": [ 2026-02-24T18:16:57.227 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stderr:report 2419369403 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: "op": "set_chooseleaf_tries", 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: "num": 5 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: "op": "set_choose_tries", 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: "num": 100 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: "op": "take", 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: "item": -1, 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: "item_name": "default" 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: "op": "choose_indep", 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: "num": 0, 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: "type": "osd" 2026-02-24T18:16:57.228 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "op": "emit" 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: ] 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "tunables": { 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "choose_local_tries": 0, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "choose_local_fallback_tries": 0, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "choose_total_tries": 50, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_descend_once": 1, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_vary_r": 1, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "chooseleaf_stable": 1, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "msr_descents": 100, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "msr_collision_tries": 100, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "straw_calc_version": 1, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "allowed_bucket_algs": 54, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "profile": "jewel", 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "optimal_tunables": 1, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "legacy_tunables": 0, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "minimum_required_version": "jewel", 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables": 1, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables2": 1, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "has_v2_rules": 1, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables3": 1, 2026-02-24T18:16:57.229 INFO:tasks.workunit.client.0.trial172.stdout: "has_v3_rules": 0, 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "has_v4_buckets": 1, 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "require_feature_tunables5": 1, 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "has_v5_rules": 0, 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "has_msr_rules": 0 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "choose_args": {} 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "fsmap": { 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 1, 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "btime": "2026-02-24T18:14:58:882446+0000", 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "default_fscid": -1, 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "compat": { 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "compat": {}, 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "ro_compat": {}, 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "incompat": { 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "feature_1": "base v0.20", 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "feature_2": "client writeable ranges", 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "feature_3": "default file layouts on dirs", 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "feature_4": "dir inode in separate object", 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "feature_5": "mds uses versioned encoding", 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "feature_8": "no anchor table", 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "feature_9": "file layout v2", 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "feature_10": "snaprealm v2", 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "feature_11": "minor log segments", 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: "feature_12": "quiesce subvolumes" 2026-02-24T18:16:57.230 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "feature_flags": { 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "enable_multiple": true, 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "ever_enabled_multiple": true 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "standbys": [], 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "filesystems": [] 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "mdsmap_first_committed": 1, 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "mdsmap_last_committed": 1, 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "auth": { 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "first_committed": 1, 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "last_committed": 5, 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "num_secrets": 3 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg": 1, 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_active": 1, 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_unknown": 0, 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "num_osd": 3, 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "pool_sum": { 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "stat_sum": { 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes": 52200, 2026-02-24T18:16:57.231 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects": 101, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_clones": 1, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_copies": 303, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_degraded": 101, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_misplaced": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_unfound": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_dirty": 101, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_whiteouts": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_read": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_read_kb": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_write": 101, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_write_kb": 101, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_scrub_errors": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_deep_scrub_errors": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_recovered": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_recovered": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_keys_recovered": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_omap": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_kb": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict": 0, 2026-02-24T18:16:57.232 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_kb": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "num_promote": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_high": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_low": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_some": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_full": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_pinned": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "num_legacy_snapsets": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "num_large_omap_objects": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_manifest": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_bytes": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_keys": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_repaired": 0 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "store_stats": { 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "total": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "available": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 0, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 0 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "log_size": 18, 2026-02-24T18:16:57.233 INFO:tasks.workunit.client.0.trial172.stdout: "ondisk_log_size": 18, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "up": 3, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "acting": 3, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "num_store_stats": 0 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "osd_sum": { 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 0, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 0, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "num_pgs": 3, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "num_osds": 3, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_osds": 3, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_omap_osds": 3, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "kb": 314572800, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used": 85076, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_data": 2276, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_omap": 31, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_meta": 82720, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "kb_avail": 314487724, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "statfs": { 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "total": 322122547200, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "available": 322035429376, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 2330624, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 1594693, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 32252, 2026-02-24T18:16:57.234 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 84705796 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: "hb_peers": [], 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: "snap_trim_queue_len": 0, 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: "num_snap_trimming": 0, 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: "num_shards_repaired": 0, 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: "op_queue_age_hist": { 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: "histogram": [], 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: "upper_bound": 1 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: "perf_stat": { 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ms": 0, 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ms": 0, 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ns": 0, 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ns": 0 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: "alerts": [], 2026-02-24T18:16:57.235 INFO:tasks.workunit.client.0.trial172.stdout: "network_ping_times": [] 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "osd_sum_by_class": { 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "ssd": { 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "up_from": 0, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 0, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "num_pgs": 3, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "num_osds": 3, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_osds": 3, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "num_per_pool_omap_osds": 3, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "kb": 314572800, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used": 85076, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_data": 2276, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_omap": 31, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "kb_used_meta": 82720, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "kb_avail": 314487724, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "statfs": { 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "total": 322122547200, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "available": 322035429376, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 2330624, 2026-02-24T18:16:57.236 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 1594693, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 32252, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 84705796 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "hb_peers": [], 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "snap_trim_queue_len": 0, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "num_snap_trimming": 0, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "num_shards_repaired": 0, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "op_queue_age_hist": { 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "histogram": [], 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "upper_bound": 1 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "perf_stat": { 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ms": 0, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ms": 0, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "commit_latency_ns": 0, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "apply_latency_ns": 0 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "alerts": [], 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "network_ping_times": [] 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "pool_stats": [ 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": 1, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg": 1, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "stat_sum": { 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes": 52200, 2026-02-24T18:16:57.237 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects": 101, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_clones": 1, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_object_copies": 303, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_missing": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_degraded": 101, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_misplaced": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_unfound": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_dirty": 101, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_whiteouts": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_read": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_read_kb": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_write": 101, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_write_kb": 101, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_scrub_errors": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_deep_scrub_errors": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_recovered": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_recovered": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_keys_recovered": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_omap": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_kb": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_kb": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_promote": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_high": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_flush_mode_low": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_some": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_evict_mode_full": 0, 2026-02-24T18:16:57.238 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_pinned": 0, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "num_legacy_snapsets": 0, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "num_large_omap_objects": 0, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_manifest": 0, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_bytes": 0, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "num_omap_keys": 0, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "num_objects_repaired": 0 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "store_stats": { 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "total": 0, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "available": 0, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "internally_reserved": 0, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "allocated": 1241088, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "data_stored": 1241088, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed": 0, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_allocated": 0, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "data_compressed_original": 0, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "omap_allocated": 0, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "internal_metadata": 0 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "log_size": 18, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "ondisk_log_size": 18, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "up": 3, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "acting": 3, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "num_store_stats": 3 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "osd_stats": [ 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 193273528329 2026-02-24T18:16:57.239 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 42949672975 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "seq": 210453397513 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_by_state": [ 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "state": "active+undersized+degraded", 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "num": 1 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "pool_pg_unavailable_map": [ 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "poolid": "1", 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "pgs": [] 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "num_pg_by_osd": [ 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2147483647, 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 1, 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 2, 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 1, 2026-02-24T18:16:57.240 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 0, 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 1, 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 1, 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "osd": 1, 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "num_primary_pg": 0, 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "num_acting_pg": 0, 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "num_up_not_acting_pg": 0 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snaps": [ 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: { 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "pool": 1, 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "purged_snaps": {} 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: ], 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "servicemap": { 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "epoch": 3, 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "modified": "2026-02-24T18:16:07.341425+0000", 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "services": { 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "osd": { 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "daemons": { 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "summary": "", 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "0": { 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "start_epoch": 0, 2026-02-24T18:16:57.241 INFO:tasks.workunit.client.0.trial172.stdout: "start_stamp": "0.000000", 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "gid": 0, 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "metadata": {}, 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "task_status": {} 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "2": { 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "start_epoch": 0, 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "start_stamp": "0.000000", 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "gid": 0, 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "metadata": {}, 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "task_status": {} 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: }, 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "mgrstat_first_committed": 1, 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "mgrstat_last_committed": 59, 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "logm_first_committed": 1, 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "logm_last_committed": 317, 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "paxos": { 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "first_committed": 1, 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "last_committed": 436, 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "last_pn": 0, 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: "accepted_pn": 0 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout: } 2026-02-24T18:16:57.242 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:16:57.243 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-24T18:16:57.243 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:341: TEST_divergent_ec: echo 'delay recovery' 2026-02-24T18:16:57.243 INFO:tasks.workunit.client.0.trial172.stdout:delay recovery 2026-02-24T18:16:57.243 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:342: TEST_divergent_ec: ceph pg dump pgs 2026-02-24T18:16:57.400 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:16:57.400 INFO:tasks.workunit.client.0.trial172.stdout:1.0 101 0 101 0 0 52200 0 0 18 84 18 active+undersized+degraded 2026-02-24T18:16:24.770576+0000 42'102 50:144 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-24T18:15:12.387431+0000 0'0 2026-02-24T18:15:12.387431+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:34:07.768062+0000 0 0 2026-02-24T18:16:57.400 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:16:57.400 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:16:57.401 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:16:57.410 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:343: TEST_divergent_ec: for i in $non_divergent 2026-02-24T18:16:57.411 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: get_asok_path osd.0 2026-02-24T18:16:57.411 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T18:16:57.411 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T18:16:57.411 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:16:57.411 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:16:57.412 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:16:57.412 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.0.asok 2026-02-24T18:16:57.413 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: CEPH_ARGS= 2026-02-24T18:16:57.413 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.0.asok set_recovery_delay 100000 2026-02-24T18:16:57.469 INFO:tasks.workunit.client.0.trial172.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-24T18:16:57.475 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:343: TEST_divergent_ec: for i in $non_divergent 2026-02-24T18:16:57.476 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: get_asok_path osd.2 2026-02-24T18:16:57.476 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-24T18:16:57.476 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-24T18:16:57.476 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:16:57.476 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:16:57.476 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:16:57.477 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.2.asok 2026-02-24T18:16:57.477 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: CEPH_ARGS= 2026-02-24T18:16:57.477 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.2.asok set_recovery_delay 100000 2026-02-24T18:16:57.532 INFO:tasks.workunit.client.0.trial172.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-24T18:16:57.538 INFO:tasks.workunit.client.0.trial172.stdout:revive divergent 1 2026-02-24T18:16:57.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:349: TEST_divergent_ec: echo 'revive divergent 1' 2026-02-24T18:16:57.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:350: TEST_divergent_ec: ceph pg dump pgs 2026-02-24T18:16:57.755 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:16:57.756 INFO:tasks.workunit.client.0.trial172.stdout:1.0 101 0 101 0 0 52200 0 0 18 84 18 active+undersized+degraded 2026-02-24T18:16:24.770576+0000 42'102 50:144 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-24T18:15:12.387431+0000 0'0 2026-02-24T18:15:12.387431+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:34:07.768062+0000 0 0 2026-02-24T18:16:57.756 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:16:57.756 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:16:57.756 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:16:57.766 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:351: TEST_divergent_ec: ceph osd set noup 2026-02-24T18:16:58.084 INFO:tasks.workunit.client.0.trial172.stderr:noup is set 2026-02-24T18:16:58.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:352: TEST_divergent_ec: activate_osd td/divergent-priors 1 2026-02-24T18:16:58.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:16:58.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:16:58.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T18:16:58.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:16:58.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-24T18:16:58.101 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:16:58.101 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:16:58.101 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:16:58.101 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:16:58.101 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-24T18:16:58.101 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-24T18:16:58.101 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:16:58.101 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:16:58.101 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:16:58.101 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:16:58.101 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:16:58.101 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:16:58.102 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:16:58.102 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:16:58.102 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:16:58.102 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:16:58.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:16:58.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:16:58.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:16:58.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:16:58.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:16:58.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:16:58.103 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:16:58.103 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:16:58.103 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:16:58.103 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:16:58.103 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-24T18:16:58.103 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:16:58.104 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T18:16:58.105 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:16:58.105 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-24T18:16:58.105 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T18:16:58.105 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:16:58.105 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:16:58.105 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:16:58.127 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:16:58.125+0000 7fe4e3c08a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:16:58.129 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:16:58.125+0000 7fe4e3c08a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:16:58.130 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:16:58.125+0000 7fe4e3c08a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:16:58.312 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: return 0 2026-02-24T18:16:58.313 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:353: TEST_divergent_ec: sleep 5 2026-02-24T18:16:58.477 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:16:58.473+0000 7fe4e3c08a40 -1 Falling back to public interface 2026-02-24T18:16:59.141 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:16:59.137+0000 7fe4e3c08a40 -1 osd.1 30 log_to_monitors true 2026-02-24T18:17:03.314 INFO:tasks.workunit.client.0.trial172.stdout:delay recovery divergent 2026-02-24T18:17:03.314 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:355: TEST_divergent_ec: echo 'delay recovery divergent' 2026-02-24T18:17:03.314 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:356: TEST_divergent_ec: ceph pg dump pgs 2026-02-24T18:17:03.525 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:17:03.525 INFO:tasks.workunit.client.0.trial172.stdout:1.0 101 0 101 0 0 52200 0 0 18 84 18 active+undersized+degraded 2026-02-24T18:16:24.770576+0000 42'102 52:148 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-24T18:15:12.387431+0000 0'0 2026-02-24T18:15:12.387431+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:34:07.768062+0000 0 0 2026-02-24T18:17:03.525 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:17:03.525 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:17:03.525 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:17:03.535 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: get_asok_path osd.1 2026-02-24T18:17:03.535 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T18:17:03.535 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T18:17:03.535 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:17:03.535 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:03.535 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:17:03.535 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19163/ceph-osd.1.asok 2026-02-24T18:17:03.536 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: CEPH_ARGS= 2026-02-24T18:17:03.536 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.19163/ceph-osd.1.asok set_recovery_delay 100000 2026-02-24T18:17:03.594 INFO:tasks.workunit.client.0.trial172.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-24T18:17:03.600 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:359: TEST_divergent_ec: ceph osd unset noup 2026-02-24T18:17:03.921 INFO:tasks.workunit.client.0.trial172.stderr:noup is unset 2026-02-24T18:17:03.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:361: TEST_divergent_ec: wait_for_osd up 0 2026-02-24T18:17:03.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:17:03.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:17:03.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:17:03.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:17:03.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:03.933 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:17:03.934 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:17:03.934 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:03.934 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:17:04.158 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 45 up_thru 49 down_at 44 last_clean_interval [37,42) [v2:127.0.0.1:6800/2122755957,v1:127.0.0.1:6801/2122755957] [v2:127.0.0.1:6802/2122755957,v1:127.0.0.1:6803/2122755957] exists,up 93b13c7c-e44d-45ef-876f-6c2e68859b9a 2026-02-24T18:17:04.159 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:17:04.159 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:17:04.159 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:17:04.159 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:362: TEST_divergent_ec: wait_for_osd up 1 2026-02-24T18:17:04.159 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:17:04.159 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:17:04.159 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:17:04.159 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:17:04.159 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:04.159 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:17:04.160 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:17:04.160 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:04.160 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:17:04.386 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 55 up_thru 55 down_at 32 last_clean_interval [10,30) [v2:127.0.0.1:6818/2850858849,v1:127.0.0.1:6819/2850858849] [v2:127.0.0.1:6820/2850858849,v1:127.0.0.1:6821/2850858849] exists,up a1643499-a87c-4291-842c-c57767e846a8 2026-02-24T18:17:04.387 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:17:04.387 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:17:04.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:17:04.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:363: TEST_divergent_ec: wait_for_osd up 2 2026-02-24T18:17:04.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:17:04.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:17:04.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:17:04.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:17:04.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:04.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:17:04.388 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:17:04.389 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:04.389 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:17:04.602 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 49 up_thru 0 down_at 48 last_clean_interval [39,42) [v2:127.0.0.1:6810/3138611225,v1:127.0.0.1:6811/3138611225] [v2:127.0.0.1:6812/3138611225,v1:127.0.0.1:6813/3138611225] exists,up 4e270c3f-1799-484f-8fb2-c5e0604b5c26 2026-02-24T18:17:04.603 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:17:04.603 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:17:04.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:17:04.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:365: TEST_divergent_ec: ceph pg dump pgs 2026-02-24T18:17:04.812 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:17:04.812 INFO:tasks.workunit.client.0.trial172.stdout:1.0 101 0 101 0 0 52200 0 0 18 84 18 active+undersized+degraded 2026-02-24T18:16:24.770576+0000 42'102 53:150 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-24T18:15:12.387431+0000 0'0 2026-02-24T18:15:12.387431+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:34:07.768062+0000 0 0 2026-02-24T18:17:04.812 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:17:04.812 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:17:04.813 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:17:04.821 INFO:tasks.workunit.client.0.trial172.stdout:wait for peering 2026-02-24T18:17:04.821 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:366: TEST_divergent_ec: echo 'wait for peering' 2026-02-24T18:17:04.821 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:367: TEST_divergent_ec: ceph pg dump pgs 2026-02-24T18:17:05.045 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:17:05.045 INFO:tasks.workunit.client.0.trial172.stdout:1.0 101 0 101 0 0 52200 0 0 18 84 18 active+undersized+degraded 2026-02-24T18:16:24.770576+0000 42'102 53:150 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-24T18:15:12.387431+0000 0'0 2026-02-24T18:15:12.387431+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:34:07.768062+0000 0 0 2026-02-24T18:17:05.045 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:17:05.045 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:17:05.046 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:17:05.054 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:368: TEST_divergent_ec: rados -p test put foo /tmp/tmp.oW6cFhP1GX 2026-02-24T18:17:05.086 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:369: TEST_divergent_ec: rm -f /tmp/tmp.oW6cFhP1GX 2026-02-24T18:17:05.087 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:371: TEST_divergent_ec: echo 'killing divergent 1' 2026-02-24T18:17:05.087 INFO:tasks.workunit.client.0.trial172.stdout:killing divergent 1 2026-02-24T18:17:05.087 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:372: TEST_divergent_ec: ceph pg dump pgs 2026-02-24T18:17:05.296 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:17:05.297 INFO:tasks.workunit.client.0.trial172.stdout:1.0 101 0 101 0 0 52200 0 0 18 84 18 active+undersized+degraded 2026-02-24T18:16:24.770576+0000 42'102 53:150 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-24T18:15:12.387431+0000 0'0 2026-02-24T18:15:12.387431+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:34:07.768062+0000 0 0 2026-02-24T18:17:05.297 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:17:05.297 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:17:05.298 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:17:05.307 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:373: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.1 2026-02-24T18:17:05.308 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:17:05.308 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:17:05.308 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:17:05.308 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:17:05.308 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:17:05.416 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:17:05.416 INFO:tasks.workunit.client.0.trial172.stdout:reviving divergent 1 2026-02-24T18:17:05.417 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:375: TEST_divergent_ec: echo 'reviving divergent 1' 2026-02-24T18:17:05.417 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:376: TEST_divergent_ec: ceph pg dump pgs 2026-02-24T18:17:05.617 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:17:05.617 INFO:tasks.workunit.client.0.trial172.stdout:1.0 101 0 0 0 0 51712 0 0 14 90 14 peering 2026-02-24T18:17:03.921091+0000 30'104 56:131 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:15:12.387431+0000 0'0 2026-02-24T18:15:12.387431+0000 0 0 no scrub is scheduled 0 0 2026-02-24T18:17:05.617 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:17:05.617 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:17:05.617 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:377: TEST_divergent_ec: activate_osd td/divergent-priors 1 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:17:05.627 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:17:05.628 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:17:05.628 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:05.628 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:17:05.628 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:17:05.628 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' 2026-02-24T18:17:05.628 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:17:05.628 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-24T18:17:05.628 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-24T18:17:05.628 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:17:05.628 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:17:05.628 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:17:05.629 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:17:05.629 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:17:05.629 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:17:05.629 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-24T18:17:05.629 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:17:05.629 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T18:17:05.630 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=27512777-e37f-4f16-b176-1621842bd4df --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.19163/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:17:05.630 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-24T18:17:05.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T18:17:05.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:17:05.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:17:05.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:17:05.658 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:05.653+0000 7f7e804d4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:05.659 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:05.657+0000 7f7e804d4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:05.660 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:05.657+0000 7f7e804d4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:05.848 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T18:17:05.848 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:17:05.849 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:17:05.849 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:17:05.849 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:17:05.849 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:05.849 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:17:05.850 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:17:05.850 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:05.850 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:17:06.064 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 55 up_thru 55 down_at 32 last_clean_interval [10,30) [v2:127.0.0.1:6818/2850858849,v1:127.0.0.1:6819/2850858849] [v2:127.0.0.1:6820/2850858849,v1:127.0.0.1:6821/2850858849] exists,up a1643499-a87c-4291-842c-c57767e846a8 2026-02-24T18:17:06.065 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:17:06.065 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:17:06.065 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:17:06.065 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:379: TEST_divergent_ec: sleep 20 2026-02-24T18:17:06.096 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:06.093+0000 7f7e804d4a40 -1 Falling back to public interface 2026-02-24T18:17:06.635 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:06.633+0000 7f7e804d4a40 -1 osd.1 56 log_to_monitors true 2026-02-24T18:17:26.066 INFO:tasks.workunit.client.0.trial172.stdout:allowing recovery 2026-02-24T18:17:26.067 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:381: TEST_divergent_ec: echo 'allowing recovery' 2026-02-24T18:17:26.067 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:382: TEST_divergent_ec: ceph pg dump pgs 2026-02-24T18:17:26.275 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:17:26.276 INFO:tasks.workunit.client.0.trial172.stdout:1.0 102 0 0 0 0 52712 0 0 13 90 13 active+clean 2026-02-24T18:17:07.000226+0000 56'103 60:162 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:15:12.387431+0000 0'0 2026-02-24T18:15:12.387431+0000 0 0 periodic scrub scheduled @ 2026-02-25T22:40:52.196152+0000 0 0 2026-02-24T18:17:26.276 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:17:26.276 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:17:26.277 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:17:26.286 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:384: TEST_divergent_ec: for i in $osds 2026-02-24T18:17:26.286 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:386: TEST_divergent_ec: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-24T18:17:26.360 INFO:tasks.workunit.client.0.trial172.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-24T18:17:26.368 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:384: TEST_divergent_ec: for i in $osds 2026-02-24T18:17:26.368 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:386: TEST_divergent_ec: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-24T18:17:26.452 INFO:tasks.workunit.client.0.trial172.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-24T18:17:26.462 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:384: TEST_divergent_ec: for i in $osds 2026-02-24T18:17:26.462 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:386: TEST_divergent_ec: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-24T18:17:26.536 INFO:tasks.workunit.client.0.trial172.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-24T18:17:26.544 INFO:tasks.workunit.client.0.trial172.stdout:reading divergent objects 2026-02-24T18:17:26.544 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:389: TEST_divergent_ec: echo 'reading divergent objects' 2026-02-24T18:17:26.544 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: ceph pg dump pgs 2026-02-24T18:17:26.759 INFO:tasks.workunit.client.0.trial172.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T18:17:26.759 INFO:tasks.workunit.client.0.trial172.stdout:1.0 102 0 0 0 0 52712 0 0 13 90 13 active+clean 2026-02-24T18:17:07.000226+0000 56'103 60:162 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T18:15:12.387431+0000 0'0 2026-02-24T18:15:12.387431+0000 0 0 periodic scrub scheduled @ 2026-02-25T22:40:52.196152+0000 0 0 2026-02-24T18:17:26.759 INFO:tasks.workunit.client.0.trial172.stdout: 2026-02-24T18:17:26.759 INFO:tasks.workunit.client.0.trial172.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T18:17:26.759 INFO:tasks.workunit.client.0.trial172.stderr:dumped pgs 2026-02-24T18:17:26.769 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:391: TEST_divergent_ec: expr 2 + 2 2026-02-24T18:17:26.770 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:391: TEST_divergent_ec: seq 1 4 2026-02-24T18:17:26.771 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:391: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:17:26.771 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:393: TEST_divergent_ec: rados -p test get existing_1 td/divergent-priors/existing 2026-02-24T18:17:26.793 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:391: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:17:26.793 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:393: TEST_divergent_ec: rados -p test get existing_2 td/divergent-priors/existing 2026-02-24T18:17:26.824 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:391: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:17:26.824 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:393: TEST_divergent_ec: rados -p test get existing_3 td/divergent-priors/existing 2026-02-24T18:17:26.850 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:391: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-24T18:17:26.850 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:393: TEST_divergent_ec: rados -p test get existing_4 td/divergent-priors/existing 2026-02-24T18:17:26.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:395: TEST_divergent_ec: rm -f td/divergent-priors/existing 2026-02-24T18:17:26.877 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:397: TEST_divergent_ec: find td/divergent-priors -name '*osd*log' 2026-02-24T18:17:26.879 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:397: TEST_divergent_ec: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-24T18:17:26.882 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 20 _merge_object_divergent_entries: merging hoid 1:7a80c0d5:::existing_99:1 entries: 30'103 (29'99) clone 1:7a80c0d5:::existing_99:1 by unknown.0.0:0 2026-02-24T18:15:25.889370+0000 0 snaps [1] ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-24T18:17:26.883 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 20 _merge_object_divergent_entries: keeping 30'103 (29'99) clone 1:7a80c0d5:::existing_99:1 by unknown.0.0:0 2026-02-24T18:15:25.889370+0000 0 snaps [1] ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-24T18:17:26.883 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-24T18:17:26.883 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 prior_version: 29'99 first_divergent_update: 30'103 last_divergent_update: 30'103 2026-02-24T18:17:26.883 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 has no more recent entries in log 2026-02-24T18:17:26.883 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 prior_version or op type indicates creation, deleting 2026-02-24T18:17:26.883 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 20 _merge_object_divergent_entries: merging hoid 1:7a80c0d5:::existing_99:head entries: 30'104 (29'99) modify 1:7a80c0d5:::existing_99:head by client.4565.0:1 2026-02-24T18:15:30.833139+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: true, new_object: false 2026-02-24T18:17:26.883 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 20 _merge_object_divergent_entries: keeping 30'104 (29'99) modify 1:7a80c0d5:::existing_99:head by client.4565.0:1 2026-02-24T18:15:30.833139+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: true, new_object: false 2026-02-24T18:17:26.883 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-24T18:17:26.883 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head prior_version: 29'99 first_divergent_update: 30'104 last_divergent_update: 30'104 2026-02-24T18:17:26.883 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head has no more recent entries in log 2026-02-24T18:17:26.884 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head must be rolled back or recovered, attempting to rollback 2026-02-24T18:17:26.884 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head olog_can_rollback_to: 30'104 2026-02-24T18:17:26.884 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head cannot rollback 30'104 (29'99) modify 1:7a80c0d5:::existing_99:head by client.4565.0:1 2026-02-24T18:15:30.833139+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: true, new_object: false 2026-02-24T18:17:26.884 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head cannot roll back, removing and adding to missing 2026-02-24T18:17:26.885 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 20 _merge_object_divergent_entries: merging hoid 1:c156033e:::existing_100:head entries: 29'101 (29'100) modify 1:c156033e:::existing_100:head by client.4556.0:1 2026-02-24T18:15:29.764450+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false,29'102 (29'101) modify 1:c156033e:::existing_100:head by client.4562.0:1 2026-02-24T18:15:30.759946+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:17:26.885 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 20 _merge_object_divergent_entries: keeping 29'101 (29'100) modify 1:c156033e:::existing_100:head by client.4556.0:1 2026-02-24T18:15:29.764450+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:17:26.885 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 20 _merge_object_divergent_entries: keeping 29'102 (29'101) modify 1:c156033e:::existing_100:head by client.4562.0:1 2026-02-24T18:15:30.759946+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:17:26.885 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-24T18:17:26.885 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head prior_version: 29'100 first_divergent_update: 29'101 last_divergent_update: 29'102 2026-02-24T18:17:26.885 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head has no more recent entries in log 2026-02-24T18:17:26.885 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head must be rolled back or recovered, attempting to rollback 2026-02-24T18:17:26.885 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head olog_can_rollback_to: 30'104 2026-02-24T18:17:26.886 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot rollback 29'102 (29'101) modify 1:c156033e:::existing_100:head by client.4562.0:1 2026-02-24T18:15:30.759946+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-24T18:17:26.886 INFO:tasks.workunit.client.0.trial172.stdout:td/divergent-priors/osd.1.log:2026-02-24T18:17:04.465+0000 7fe4c5b8e640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot roll back, removing and adding to missing 2026-02-24T18:17:26.886 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:406: TEST_divergent_ec: find td/divergent-priors -name '*osd*log' 2026-02-24T18:17:26.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:406: TEST_divergent_ec: grep -q '_merge_object_divergent_entries.*rolled back' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-24T18:17:26.890 INFO:tasks.workunit.client.0.trial172.stdout:success 2026-02-24T18:17:26.890 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:411: TEST_divergent_ec: echo success 2026-02-24T18:17:26.890 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:413: TEST_divergent_ec: delete_pool test 2026-02-24T18:17:26.890 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=test 2026-02-24T18:17:26.891 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-24T18:17:27.162 INFO:tasks.workunit.client.0.trial172.stderr:pool 'test' does not exist 2026-02-24T18:17:27.173 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:414: TEST_divergent_ec: kill_daemons td/divergent-priors 2026-02-24T18:17:27.175 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:17:27.175 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:17:27.175 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:17:27.175 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:17:27.175 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:17:32.491 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:17:32.491 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-24T18:17:32.491 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-24T18:17:32.491 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:17:32.491 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-24T18:17:32.492 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:17:32.492 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:17:32.492 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:17:32.493 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:17:32.493 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:17:32.499 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:17:32.499 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:17:32.500 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:17:32.500 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:17:32.501 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:17:32.501 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:17:32.502 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:17:32.503 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:17:32.503 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:17:32.504 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:17:32.504 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:17:32.505 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:17:32.506 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:17:32.506 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-24T18:17:32.529 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:17:32.529 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:32.529 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:17:32.529 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19163 2026-02-24T18:17:32.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:17:32.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:17:32.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2429: main: code=0 2026-02-24T18:17:32.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2433: main: teardown td/divergent-priors 0 2026-02-24T18:17:32.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-24T18:17:32.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-24T18:17:32.530 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-24T18:17:32.531 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:17:32.531 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:17:32.531 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:17:32.531 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:17:32.531 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:17:32.534 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:17:32.534 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:17:32.535 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:17:32.536 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:17:32.537 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:17:32.537 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:17:32.538 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:17:32.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:17:32.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:17:32.539 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:17:32.539 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:17:32.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:17:32.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-24T18:17:32.541 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-24T18:17:32.543 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:17:32.543 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:32.543 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19163 2026-02-24T18:17:32.543 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19163 2026-02-24T18:17:32.546 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:17:32.546 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:17:32.546 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2434: main: return 0 2026-02-24T18:17:32.546 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T18:17:32.547 DEBUG:teuthology.orchestra.run.trial172:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T18:17:32.556 INFO:tasks.workunit:Running workunit osd/ec-error-rollforward.sh... 2026-02-24T18:17:32.556 DEBUG:teuthology.orchestra.run.trial172:workunit test osd/ec-error-rollforward.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=618c318a7a0b287ab5aeff207b7bf1d73ef45044 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh 2026-02-24T18:17:32.608 INFO:tasks.workunit.client.0.trial172.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-24T18:17:32.611 INFO:tasks.workunit.client.0.trial172.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-24T18:17:32.611 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T18:17:32.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T18:17:32.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: export PYTHONWARNINGS=ignore 2026-02-24T18:17:32.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: PYTHONWARNINGS=ignore 2026-02-24T18:17:32.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: export CEPH_CONF=/dev/null 2026-02-24T18:17:32.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: CEPH_CONF=/dev/null 2026-02-24T18:17:32.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: main: unset CEPH_ARGS 2026-02-24T18:17:32.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: main: local code 2026-02-24T18:17:32.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2428: main: run td/ec-error-rollforward 2026-02-24T18:17:32.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:6: run: local dir=td/ec-error-rollforward 2026-02-24T18:17:32.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:7: run: shift 2026-02-24T18:17:32.613 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:10: run: export CEPH_MON=127.0.0.1:7132 2026-02-24T18:17:32.613 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:10: run: CEPH_MON=127.0.0.1:7132 2026-02-24T18:17:32.613 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:11: run: export CEPH_ARGS 2026-02-24T18:17:32.613 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:12: run: uuidgen 2026-02-24T18:17:32.614 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:12: run: CEPH_ARGS+='--fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none ' 2026-02-24T18:17:32.614 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:13: run: CEPH_ARGS+='--mon-host=127.0.0.1:7132 ' 2026-02-24T18:17:32.615 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:14: run: export margin=10 2026-02-24T18:17:32.615 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:14: run: margin=10 2026-02-24T18:17:32.615 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:15: run: export objects=200 2026-02-24T18:17:32.615 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:15: run: objects=200 2026-02-24T18:17:32.615 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:16: run: export poolname=test 2026-02-24T18:17:32.615 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:16: run: poolname=test 2026-02-24T18:17:32.615 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: set 2026-02-24T18:17:32.615 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-24T18:17:32.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: local funcs=TEST_ec_error_rollforward 2026-02-24T18:17:32.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:19: run: for func in $funcs 2026-02-24T18:17:32.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:20: run: setup td/ec-error-rollforward 2026-02-24T18:17:32.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ec-error-rollforward 2026-02-24T18:17:32.617 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ec-error-rollforward 2026-02-24T18:17:32.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ec-error-rollforward 2026-02-24T18:17:32.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:17:32.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ec-error-rollforward KILL 2026-02-24T18:17:32.618 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:17:32.618 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:17:32.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:17:32.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:17:32.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:17:32.619 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:17:32.619 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:17:32.619 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:17:32.619 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:17:32.620 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:17:32.620 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:17:32.621 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:17:32.622 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:17:32.622 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:17:32.622 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:17:32.622 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:17:32.623 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:17:32.624 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:17:32.625 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ec-error-rollforward 2026-02-24T18:17:32.626 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:17:32.626 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:32.626 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:32.626 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.77100 2026-02-24T18:17:32.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:17:32.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:17:32.628 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ec-error-rollforward 2026-02-24T18:17:32.629 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T18:17:32.629 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:32.629 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:32.629 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.77100 2026-02-24T18:17:32.631 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T18:17:32.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-24T18:17:32.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-24T18:17:32.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T18:17:32.632 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ec-error-rollforward 1' TERM HUP INT 2026-02-24T18:17:32.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:21: run: TEST_ec_error_rollforward td/ec-error-rollforward 2026-02-24T18:17:32.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:27: TEST_ec_error_rollforward: local dir=td/ec-error-rollforward 2026-02-24T18:17:32.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:28: TEST_ec_error_rollforward: run_mon td/ec-error-rollforward a 2026-02-24T18:17:32.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ec-error-rollforward 2026-02-24T18:17:32.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T18:17:32.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T18:17:32.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T18:17:32.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ec-error-rollforward/a 2026-02-24T18:17:32.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ec-error-rollforward/a --run-dir=td/ec-error-rollforward 2026-02-24T18:17:32.656 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T18:17:32.657 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:17:32.657 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:17:32.657 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:17:32.657 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:32.657 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:32.657 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:32.657 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ec-error-rollforward/a '--log-file=td/ec-error-rollforward/$name.log' '--admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' --mon-cluster-log-file=td/ec-error-rollforward/log --run-dir=td/ec-error-rollforward '--pid-file=td/ec-error-rollforward/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T18:17:32.684 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T18:17:32.686 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T18:17:32.686 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:17:32.686 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:17:32.686 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T18:17:32.686 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T18:17:32.686 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:17:32.686 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:17:32.687 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:17:32.687 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:17:32.688 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:32.688 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:32.688 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.77100/ceph-mon.a.asok 2026-02-24T18:17:32.688 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:17:32.688 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.77100/ceph-mon.a.asok config get fsid 2026-02-24T18:17:32.744 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T18:17:32.744 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:17:32.745 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:17:32.745 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T18:17:32.745 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T18:17:32.745 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:17:32.745 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:17:32.745 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:17:32.745 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:17:32.745 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:32.745 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:32.745 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.77100/ceph-mon.a.asok 2026-02-24T18:17:32.746 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:17:32.746 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.77100/ceph-mon.a.asok config get mon_host 2026-02-24T18:17:32.801 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:29: TEST_ec_error_rollforward: run_mgr td/ec-error-rollforward x 2026-02-24T18:17:32.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ec-error-rollforward 2026-02-24T18:17:32.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T18:17:32.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T18:17:32.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T18:17:32.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ec-error-rollforward/x 2026-02-24T18:17:32.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T18:17:32.915 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T18:17:32.916 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:17:32.916 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:17:32.916 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:17:32.916 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:32.916 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:32.916 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:32.917 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:17:32.917 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ec-error-rollforward/x '--log-file=td/ec-error-rollforward/$name.log' '--admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' --run-dir=td/ec-error-rollforward '--pid-file=td/ec-error-rollforward/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:17:32.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:30: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 0 2026-02-24T18:17:32.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ec-error-rollforward 2026-02-24T18:17:32.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:17:32.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T18:17:32.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:17:32.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ec-error-rollforward/0 2026-02-24T18:17:32.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-24T18:17:32.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:17:32.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:17:32.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:17:32.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/0' 2026-02-24T18:17:32.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/0/journal' 2026-02-24T18:17:32.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:17:32.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:17:32.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-24T18:17:32.939 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:17:32.939 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:17:32.939 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:17:32.939 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:17:32.939 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:32.940 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:32.940 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:32.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:32.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:17:32.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:17:32.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:17:32.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-24T18:17:32.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-24T18:17:32.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:17:32.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:17:32.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:17:32.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:17:32.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:17:32.941 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:17:32.941 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ec-error-rollforward/0 2026-02-24T18:17:32.941 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:17:32.941 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=12f04734-90b3-4198-adfa-0cb12e68410d 2026-02-24T18:17:32.941 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 12f04734-90b3-4198-adfa-0cb12e68410d' 2026-02-24T18:17:32.942 INFO:tasks.workunit.client.0.trial172.stdout:add osd0 12f04734-90b3-4198-adfa-0cb12e68410d 2026-02-24T18:17:32.942 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:17:32.949 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA8651pPu9mOBAAsAzxpdPeEZbI5np3RcZPFA== 2026-02-24T18:17:32.949 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA8651pPu9mOBAAsAzxpdPeEZbI5np3RcZPFA=="}' 2026-02-24T18:17:32.949 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 12f04734-90b3-4198-adfa-0cb12e68410d -i td/ec-error-rollforward/0/new.json 2026-02-24T18:17:33.059 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:17:33.066 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ec-error-rollforward/0/new.json 2026-02-24T18:17:33.067 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA8651pPu9mOBAAsAzxpdPeEZbI5np3RcZPFA== --osd-uuid 12f04734-90b3-4198-adfa-0cb12e68410d 2026-02-24T18:17:33.083 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:33.080+0000 7f8d15589a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:33.084 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:33.080+0000 7f8d15589a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:33.085 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:33.080+0000 7f8d15589a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:33.085 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:33.080+0000 7f8d15589a40 -1 bdev(0x55c0b986c400 td/ec-error-rollforward/0/block) open stat got: (1) Operation not permitted 2026-02-24T18:17:33.085 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:33.080+0000 7f8d15589a40 -1 bluestore(td/ec-error-rollforward/0) _read_fsid unparsable uuid 2026-02-24T18:17:33.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ec-error-rollforward/0/keyring 2026-02-24T18:17:33.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:17:33.778 INFO:tasks.workunit.client.0.trial172.stdout:adding osd0 key to auth repository 2026-02-24T18:17:33.778 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T18:17:33.778 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ec-error-rollforward/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:17:33.890 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:17:33.891 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T18:17:33.892 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:17:33.892 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:17:33.892 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:17:33.892 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:17:33.905 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:33.900+0000 7f1e6c1fea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:33.906 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:33.900+0000 7f1e6c1fea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:33.907 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:33.904+0000 7f1e6c1fea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:34.004 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T18:17:34.004 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:17:34.004 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:17:34.004 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:17:34.004 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:17:34.004 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:34.004 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:17:34.005 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:17:34.005 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:34.005 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:17:34.113 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:17:34.329 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:34.324+0000 7f1e6c1fea40 -1 Falling back to public interface 2026-02-24T18:17:35.020 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:35.016+0000 7f1e6c1fea40 -1 osd.0 0 log_to_monitors true 2026-02-24T18:17:35.114 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:17:35.114 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:17:35.114 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:35.114 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:17:35.115 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:17:35.116 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:35.245 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:17:36.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:17:36.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:36.257 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:17:36.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:17:36.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:36.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:17:36.458 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/170283968,v1:127.0.0.1:6801/170283968] [v2:127.0.0.1:6802/170283968,v1:127.0.0.1:6803/170283968] exists,up 12f04734-90b3-4198-adfa-0cb12e68410d 2026-02-24T18:17:36.461 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:17:36.461 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:17:36.461 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:17:36.461 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:31: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 1 2026-02-24T18:17:36.462 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ec-error-rollforward 2026-02-24T18:17:36.462 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:17:36.462 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T18:17:36.462 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:17:36.462 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ec-error-rollforward/1 2026-02-24T18:17:36.462 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-24T18:17:36.462 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:17:36.462 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:17:36.462 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:17:36.462 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/1' 2026-02-24T18:17:36.462 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/1/journal' 2026-02-24T18:17:36.463 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:17:36.463 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:17:36.463 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-24T18:17:36.463 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:17:36.463 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:17:36.463 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:17:36.463 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:17:36.463 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:36.463 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:36.463 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:36.463 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:36.463 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:17:36.463 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:17:36.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:17:36.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-24T18:17:36.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-24T18:17:36.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:17:36.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:17:36.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:17:36.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:17:36.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:17:36.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:17:36.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ec-error-rollforward/1 2026-02-24T18:17:36.465 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:17:36.465 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6682f6f6-9b73-426c-bb26-4231951004f8 2026-02-24T18:17:36.465 INFO:tasks.workunit.client.0.trial172.stdout:add osd1 6682f6f6-9b73-426c-bb26-4231951004f8 2026-02-24T18:17:36.466 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 6682f6f6-9b73-426c-bb26-4231951004f8' 2026-02-24T18:17:36.466 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:17:36.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBA651pu8wWHBAAObOL0MaZyD2GE8m7SW7ytQ== 2026-02-24T18:17:36.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBA651pu8wWHBAAObOL0MaZyD2GE8m7SW7ytQ=="}' 2026-02-24T18:17:36.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6682f6f6-9b73-426c-bb26-4231951004f8 -i td/ec-error-rollforward/1/new.json 2026-02-24T18:17:36.701 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:17:36.710 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ec-error-rollforward/1/new.json 2026-02-24T18:17:36.711 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBA651pu8wWHBAAObOL0MaZyD2GE8m7SW7ytQ== --osd-uuid 6682f6f6-9b73-426c-bb26-4231951004f8 2026-02-24T18:17:36.727 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:36.724+0000 7f5943268a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:36.729 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:36.724+0000 7f5943268a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:36.730 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:36.724+0000 7f5943268a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:36.730 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:36.724+0000 7f5943268a40 -1 bdev(0x55be54168400 td/ec-error-rollforward/1/block) open stat got: (1) Operation not permitted 2026-02-24T18:17:36.731 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:36.724+0000 7f5943268a40 -1 bluestore(td/ec-error-rollforward/1) _read_fsid unparsable uuid 2026-02-24T18:17:37.411 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ec-error-rollforward/1/keyring 2026-02-24T18:17:37.411 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:17:37.412 INFO:tasks.workunit.client.0.trial172.stdout:adding osd1 key to auth repository 2026-02-24T18:17:37.412 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T18:17:37.412 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ec-error-rollforward/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:17:37.707 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:17:37.708 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T18:17:37.708 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:17:37.708 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:17:37.708 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:17:37.708 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:17:37.730 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:37.724+0000 7f612d813a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:37.732 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:37.728+0000 7f612d813a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:37.733 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:37.728+0000 7f612d813a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:37.922 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T18:17:37.922 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:17:37.922 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:17:37.922 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:17:37.922 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:17:37.922 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:37.923 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:17:37.923 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:17:37.923 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:37.923 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:17:38.101 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:38.096+0000 7f612d813a40 -1 Falling back to public interface 2026-02-24T18:17:38.139 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:17:38.772 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:38.768+0000 7f612d813a40 -1 osd.1 0 log_to_monitors true 2026-02-24T18:17:39.140 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:17:39.140 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:39.140 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:17:39.140 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:17:39.140 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:39.140 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:17:39.363 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:17:40.364 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:17:40.365 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:40.365 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:17:40.365 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:17:40.366 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:40.366 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:17:40.578 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2193855253,v1:127.0.0.1:6811/2193855253] [v2:127.0.0.1:6812/2193855253,v1:127.0.0.1:6813/2193855253] exists,up 6682f6f6-9b73-426c-bb26-4231951004f8 2026-02-24T18:17:40.581 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:17:40.581 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:17:40.581 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:17:40.581 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:32: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 2 2026-02-24T18:17:40.581 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ec-error-rollforward 2026-02-24T18:17:40.581 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:17:40.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T18:17:40.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:17:40.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ec-error-rollforward/2 2026-02-24T18:17:40.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-24T18:17:40.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:17:40.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:17:40.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:17:40.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/2' 2026-02-24T18:17:40.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/2/journal' 2026-02-24T18:17:40.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:17:40.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:17:40.583 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-24T18:17:40.583 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:17:40.583 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:17:40.583 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:17:40.583 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:17:40.583 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:40.583 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:40.583 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:40.583 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:40.583 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:17:40.583 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:17:40.583 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:17:40.584 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-24T18:17:40.584 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-24T18:17:40.584 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:17:40.584 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:17:40.584 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:17:40.584 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:17:40.584 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:17:40.584 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:17:40.584 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ec-error-rollforward/2 2026-02-24T18:17:40.585 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:17:40.585 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7ae111cd-9052-466e-a489-3a58b099e217 2026-02-24T18:17:40.586 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 7ae111cd-9052-466e-a489-3a58b099e217' 2026-02-24T18:17:40.586 INFO:tasks.workunit.client.0.trial172.stdout:add osd2 7ae111cd-9052-466e-a489-3a58b099e217 2026-02-24T18:17:40.586 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:17:40.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBE651pfz1AIxAAt11m82j2ezk7I6zWzDNjMg== 2026-02-24T18:17:40.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBE651pfz1AIxAAt11m82j2ezk7I6zWzDNjMg=="}' 2026-02-24T18:17:40.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7ae111cd-9052-466e-a489-3a58b099e217 -i td/ec-error-rollforward/2/new.json 2026-02-24T18:17:40.819 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:17:40.828 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ec-error-rollforward/2/new.json 2026-02-24T18:17:40.828 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBE651pfz1AIxAAt11m82j2ezk7I6zWzDNjMg== --osd-uuid 7ae111cd-9052-466e-a489-3a58b099e217 2026-02-24T18:17:40.844 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:40.840+0000 7f882675ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:40.845 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:40.840+0000 7f882675ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:40.846 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:40.840+0000 7f882675ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:40.847 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:40.840+0000 7f882675ca40 -1 bdev(0x562118fae400 td/ec-error-rollforward/2/block) open stat got: (1) Operation not permitted 2026-02-24T18:17:40.847 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:40.844+0000 7f882675ca40 -1 bluestore(td/ec-error-rollforward/2) _read_fsid unparsable uuid 2026-02-24T18:17:41.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ec-error-rollforward/2/keyring 2026-02-24T18:17:41.474 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:17:41.475 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T18:17:41.475 INFO:tasks.workunit.client.0.trial172.stdout:adding osd2 key to auth repository 2026-02-24T18:17:41.475 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ec-error-rollforward/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:17:41.769 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:17:41.770 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T18:17:41.770 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:17:41.770 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:17:41.770 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:17:41.770 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:17:41.792 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:41.788+0000 7fc0af934a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:41.794 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:41.788+0000 7fc0af934a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:41.795 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:41.792+0000 7fc0af934a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:41.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T18:17:41.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:17:41.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:17:41.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:17:41.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:17:41.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:41.983 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:17:41.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:17:41.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:41.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:17:42.202 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:17:42.277 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:42.272+0000 7fc0af934a40 -1 Falling back to public interface 2026-02-24T18:17:42.898 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:42.892+0000 7fc0af934a40 -1 osd.2 0 log_to_monitors true 2026-02-24T18:17:43.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:17:43.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:43.204 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:17:43.204 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:17:43.204 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:43.205 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:17:43.428 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:17:44.429 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:17:44.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:44.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:17:44.430 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:17:44.431 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:44.431 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:17:44.643 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3419666880,v1:127.0.0.1:6819/3419666880] [v2:127.0.0.1:6820/3419666880,v1:127.0.0.1:6821/3419666880] exists,up 7ae111cd-9052-466e-a489-3a58b099e217 2026-02-24T18:17:44.644 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:33: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 3 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ec-error-rollforward 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ec-error-rollforward/3 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/3' 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/3/journal' 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:17:44.645 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:17:44.646 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:17:44.646 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:44.646 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:44.646 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:44.646 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:44.646 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:17:44.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:17:44.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:17:44.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-24T18:17:44.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-24T18:17:44.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:17:44.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:17:44.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:17:44.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:17:44.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:17:44.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:17:44.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ec-error-rollforward/3 2026-02-24T18:17:44.647 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:17:44.648 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8c555b40-7ade-4e43-84ca-1aced5bfc566 2026-02-24T18:17:44.648 INFO:tasks.workunit.client.0.trial172.stdout:add osd3 8c555b40-7ade-4e43-84ca-1aced5bfc566 2026-02-24T18:17:44.648 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 8c555b40-7ade-4e43-84ca-1aced5bfc566' 2026-02-24T18:17:44.648 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:17:44.660 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBI651p5wc1JxAAngKDj9Pit+RO/aRfatIb2Q== 2026-02-24T18:17:44.660 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBI651p5wc1JxAAngKDj9Pit+RO/aRfatIb2Q=="}' 2026-02-24T18:17:44.660 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8c555b40-7ade-4e43-84ca-1aced5bfc566 -i td/ec-error-rollforward/3/new.json 2026-02-24T18:17:44.883 INFO:tasks.workunit.client.0.trial172.stdout:3 2026-02-24T18:17:44.894 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ec-error-rollforward/3/new.json 2026-02-24T18:17:44.895 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBI651p5wc1JxAAngKDj9Pit+RO/aRfatIb2Q== --osd-uuid 8c555b40-7ade-4e43-84ca-1aced5bfc566 2026-02-24T18:17:44.917 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:44.912+0000 7f704e37ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:44.919 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:44.912+0000 7f704e37ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:44.920 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:44.916+0000 7f704e37ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:44.920 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:44.916+0000 7f704e37ea40 -1 bdev(0x562060ed4400 td/ec-error-rollforward/3/block) open stat got: (1) Operation not permitted 2026-02-24T18:17:44.921 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:44.916+0000 7f704e37ea40 -1 bluestore(td/ec-error-rollforward/3) _read_fsid unparsable uuid 2026-02-24T18:17:45.586 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ec-error-rollforward/3/keyring 2026-02-24T18:17:45.587 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:17:45.587 INFO:tasks.workunit.client.0.trial172.stdout:adding osd3 key to auth repository 2026-02-24T18:17:45.588 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T18:17:45.588 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ec-error-rollforward/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:17:45.882 INFO:tasks.workunit.client.0.trial172.stdout:start osd.3 2026-02-24T18:17:45.882 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T18:17:45.882 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:17:45.882 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:17:45.882 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:17:45.883 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:17:45.904 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:45.900+0000 7f101f1aca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:45.906 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:45.900+0000 7f101f1aca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:45.907 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:45.904+0000 7f101f1aca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:46.092 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T18:17:46.092 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:17:46.092 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T18:17:46.092 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:17:46.092 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:17:46.093 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:46.093 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:17:46.093 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:17:46.093 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:46.094 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:17:46.289 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:46.284+0000 7f101f1aca40 -1 Falling back to public interface 2026-02-24T18:17:46.308 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:17:46.882 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:46.876+0000 7f101f1aca40 -1 osd.3 0 log_to_monitors true 2026-02-24T18:17:47.309 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:17:47.310 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:47.310 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:17:47.311 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:17:47.311 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:47.311 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:17:47.527 INFO:tasks.workunit.client.0.trial172.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3380363980,v1:127.0.0.1:6827/3380363980] [v2:127.0.0.1:6828/3380363980,v1:127.0.0.1:6829/3380363980] exists,up 8c555b40-7ade-4e43-84ca-1aced5bfc566 2026-02-24T18:17:47.528 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:17:47.528 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:17:47.528 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:17:47.529 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:35: TEST_ec_error_rollforward: ceph osd erasure-code-profile set ec-profile m=2 k=2 crush-failure-domain=osd 2026-02-24T18:17:47.763 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:36: TEST_ec_error_rollforward: ceph osd pool create ec 1 1 erasure ec-profile 2026-02-24T18:17:48.155 INFO:tasks.workunit.client.0.trial172.stderr:pool 'ec' already exists 2026-02-24T18:17:48.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:38: TEST_ec_error_rollforward: rados -p ec put foo /etc/passwd 2026-02-24T18:17:49.097 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:40: TEST_ec_error_rollforward: cat td/ec-error-rollforward/osd.2.pid 2026-02-24T18:17:49.098 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:40: TEST_ec_error_rollforward: kill -STOP 81285 2026-02-24T18:17:49.098 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:43: TEST_ec_error_rollforward: pids=83621 2026-02-24T18:17:49.098 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:44: TEST_ec_error_rollforward: sleep 1 2026-02-24T18:17:49.099 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:42: TEST_ec_error_rollforward: rados -p ec rm foo 2026-02-24T18:17:50.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:46: TEST_ec_error_rollforward: pids+=' 83642' 2026-02-24T18:17:50.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:45: TEST_ec_error_rollforward: rados -p ec rm a 2026-02-24T18:17:50.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:48: TEST_ec_error_rollforward: pids+=' 83643' 2026-02-24T18:17:50.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:47: TEST_ec_error_rollforward: rados -p ec rm b 2026-02-24T18:17:50.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:50: TEST_ec_error_rollforward: pids+=' 83644' 2026-02-24T18:17:50.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:51: TEST_ec_error_rollforward: sleep 1 2026-02-24T18:17:50.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:49: TEST_ec_error_rollforward: rados -p ec rm c 2026-02-24T18:17:51.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:54: TEST_ec_error_rollforward: kill_daemons td/ec-error-rollforward KILL osd 2026-02-24T18:17:51.103 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:17:51.103 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:17:51.103 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:17:51.104 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:17:51.104 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:17:51.414 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:17:51.414 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:55: TEST_ec_error_rollforward: kill 83621 83642 83643 83644 2026-02-24T18:17:51.414 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:56: TEST_ec_error_rollforward: wait 2026-02-24T18:17:51.419 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:58: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 0 2026-02-24T18:17:51.419 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ec-error-rollforward 2026-02-24T18:17:51.419 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:17:51.419 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-24T18:17:51.419 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:17:51.419 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ec-error-rollforward/0 2026-02-24T18:17:51.419 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-24T18:17:51.420 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:17:51.420 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:17:51.420 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:17:51.420 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/0' 2026-02-24T18:17:51.420 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/0/journal' 2026-02-24T18:17:51.420 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:17:51.420 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:17:51.420 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-24T18:17:51.420 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:17:51.420 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:17:51.420 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:17:51.420 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:17:51.421 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:51.421 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:51.421 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:51.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:51.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:17:51.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-24T18:17:51.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-24T18:17:51.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:17:51.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:17:51.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:17:51.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:17:51.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:17:51.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:17:51.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ec-error-rollforward/0 2026-02-24T18:17:51.423 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:17:51.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-24T18:17:51.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:17:51.424 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ec-error-rollforward/0/whoami 2026-02-24T18:17:51.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-24T18:17:51.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:17:51.425 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:17:51.425 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:17:51.444 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:51.440+0000 7f5467020a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:51.445 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:51.440+0000 7f5467020a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:51.446 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:51.440+0000 7f5467020a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:51.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-24T18:17:51.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:17:51.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:17:51.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:17:51.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:17:51.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:51.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:17:51.639 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:17:51.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:51.640 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:17:51.841 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:51.836+0000 7f5467020a40 -1 Falling back to public interface 2026-02-24T18:17:51.851 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/170283968,v1:127.0.0.1:6801/170283968] [v2:127.0.0.1:6802/170283968,v1:127.0.0.1:6803/170283968] exists,up 12f04734-90b3-4198-adfa-0cb12e68410d 2026-02-24T18:17:51.854 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:17:51.854 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:17:51.855 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:17:51.855 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:59: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 1 2026-02-24T18:17:51.855 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ec-error-rollforward 2026-02-24T18:17:51.855 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:17:51.855 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T18:17:51.855 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:17:51.855 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ec-error-rollforward/1 2026-02-24T18:17:51.855 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-24T18:17:51.855 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:17:51.855 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:17:51.856 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:17:51.856 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/1' 2026-02-24T18:17:51.856 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/1/journal' 2026-02-24T18:17:51.856 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:17:51.856 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:17:51.856 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-24T18:17:51.856 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:17:51.856 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:17:51.856 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:17:51.856 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:17:51.857 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:51.857 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:51.857 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:51.857 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:51.857 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:17:51.857 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-24T18:17:51.857 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-24T18:17:51.857 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:17:51.857 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:17:51.857 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:17:51.857 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:17:51.858 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:17:51.858 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:17:51.858 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ec-error-rollforward/1 2026-02-24T18:17:51.858 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:17:51.859 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T18:17:51.859 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:17:51.860 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ec-error-rollforward/1/whoami 2026-02-24T18:17:51.860 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T18:17:51.860 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:17:51.860 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:17:51.860 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:17:51.871 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:51.868+0000 7fb47c66ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:51.873 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:51.868+0000 7fb47c66ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:51.874 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:51.868+0000 7fb47c66ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:52.069 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T18:17:52.069 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:17:52.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:17:52.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:17:52.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:17:52.070 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:17:52.071 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:52.071 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:17:52.071 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:52.071 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:17:52.280 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 10 up_thru 22 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2193855253,v1:127.0.0.1:6811/2193855253] [v2:127.0.0.1:6812/2193855253,v1:127.0.0.1:6813/2193855253] exists,up 6682f6f6-9b73-426c-bb26-4231951004f8 2026-02-24T18:17:52.282 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:17:52.283 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:17:52.283 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:17:52.283 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:60: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 2 2026-02-24T18:17:52.283 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ec-error-rollforward 2026-02-24T18:17:52.283 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:17:52.283 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T18:17:52.283 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:17:52.283 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ec-error-rollforward/2 2026-02-24T18:17:52.283 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-24T18:17:52.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:17:52.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:17:52.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:17:52.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/2' 2026-02-24T18:17:52.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/2/journal' 2026-02-24T18:17:52.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:17:52.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:17:52.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-24T18:17:52.284 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:17:52.284 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:17:52.284 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:17:52.284 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:17:52.285 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:52.285 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:52.285 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:52.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:52.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:17:52.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-24T18:17:52.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-24T18:17:52.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:17:52.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:17:52.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:17:52.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:17:52.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:17:52.286 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:17:52.286 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ec-error-rollforward/2 2026-02-24T18:17:52.286 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:17:52.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T18:17:52.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:17:52.287 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ec-error-rollforward/2/whoami 2026-02-24T18:17:52.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T18:17:52.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:17:52.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:17:52.288 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:17:52.300 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:52.296+0000 7f1e19939a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:52.301 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:52.296+0000 7f1e19939a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:52.302 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:52.296+0000 7f1e19939a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:52.325 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:52.320+0000 7fb47c66ea40 -1 Falling back to public interface 2026-02-24T18:17:52.499 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T18:17:52.499 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:17:52.499 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:17:52.499 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:17:52.499 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:17:52.500 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:17:52.500 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:52.500 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:17:52.500 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:52.500 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:17:52.701 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:52.696+0000 7f1e19939a40 -1 Falling back to public interface 2026-02-24T18:17:52.715 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3419666880,v1:127.0.0.1:6819/3419666880] [v2:127.0.0.1:6820/3419666880,v1:127.0.0.1:6821/3419666880] exists,up 7ae111cd-9052-466e-a489-3a58b099e217 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:61: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 3 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ec-error-rollforward 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ec-error-rollforward/3 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/3' 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/3/journal' 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:17:52.717 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:17:52.718 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:17:52.718 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:52.718 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:52.718 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:52.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' 2026-02-24T18:17:52.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:17:52.719 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-24T18:17:52.719 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-24T18:17:52.719 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:17:52.719 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:17:52.719 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:17:52.719 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:17:52.719 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:17:52.719 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:17:52.719 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ec-error-rollforward/3 2026-02-24T18:17:52.719 INFO:tasks.workunit.client.0.trial172.stdout:start osd.3 2026-02-24T18:17:52.719 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-24T18:17:52.719 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=66ad27ee-d316-42a7-9f73-e895eb0ae8a1 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.77100/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:17:52.719 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ec-error-rollforward/3/whoami 2026-02-24T18:17:52.720 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-24T18:17:52.720 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:17:52.720 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:17:52.720 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:17:52.733 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:52.728+0000 7f3882b04a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:52.734 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:52.728+0000 7f3882b04a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:52.735 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:52.732+0000 7f3882b04a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:52.737 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:52.732+0000 7f5467020a40 -1 osd.0 24 log_to_monitors true 2026-02-24T18:17:52.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-24T18:17:52.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:17:52.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T18:17:52.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:17:52.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:17:52.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:52.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:17:52.938 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:17:52.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:52.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:17:53.156 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:17:53.208 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:53.204+0000 7fb47c66ea40 -1 osd.1 24 log_to_monitors true 2026-02-24T18:17:53.228 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:53.224+0000 7f3882b04a40 -1 Falling back to public interface 2026-02-24T18:17:53.592 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:53.588+0000 7f1e19939a40 -1 osd.2 23 log_to_monitors true 2026-02-24T18:17:54.057 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:54.052+0000 7f3882b04a40 -1 osd.3 24 log_to_monitors true 2026-02-24T18:17:54.157 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:17:54.157 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:54.157 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:17:54.158 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:17:54.158 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:54.158 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:17:54.371 INFO:tasks.workunit.client.0.trial172.stdout:osd.3 up in weight 1 up_from 37 up_thru 0 down_at 28 last_clean_interval [18,24) [v2:127.0.0.1:6826/1083689910,v1:127.0.0.1:6827/1083689910] [v2:127.0.0.1:6828/1083689910,v1:127.0.0.1:6829/1083689910] exists,up 8c555b40-7ade-4e43-84ca-1aced5bfc566 2026-02-24T18:17:54.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:17:54.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:17:54.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:17:54.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:63: TEST_ec_error_rollforward: wait_for_clean 2026-02-24T18:17:54.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:17:54.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:17:54.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:17:54.374 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T18:17:54.374 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:17:54.374 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:17:54.374 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:17:54.374 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:17:54.374 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:17:54.471 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T18:17:54.471 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:17:54.471 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:17:54.471 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:17:54.471 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:17:54.471 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:17:54.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:17:54.693 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:17:54.693 INFO:tasks.workunit.client.0.trial172.stderr:2 2026-02-24T18:17:54.693 INFO:tasks.workunit.client.0.trial172.stderr:3' 2026-02-24T18:17:54.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:17:54.693 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:17:54.694 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:17:54.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964116995 2026-02-24T18:17:54.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964116995 2026-02-24T18:17:54.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-115964116995' 2026-02-24T18:17:54.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:17:54.774 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:17:54.850 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=133143986179 2026-02-24T18:17:54.850 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 133143986179 2026-02-24T18:17:54.850 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-115964116995 1-133143986179' 2026-02-24T18:17:54.850 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:17:54.851 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:17:54.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=146028888067 2026-02-24T18:17:54.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 146028888067 2026-02-24T18:17:54.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-115964116995 1-133143986179 2-146028888067' 2026-02-24T18:17:54.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:17:54.932 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T18:17:55.009 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=158913789955 2026-02-24T18:17:55.009 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 158913789955 2026-02-24T18:17:55.010 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-115964116995 1-133143986179 2-146028888067 3-158913789955' 2026-02-24T18:17:55.010 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:17:55.010 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-115964116995 2026-02-24T18:17:55.010 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:17:55.010 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:17:55.011 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-115964116995 2026-02-24T18:17:55.011 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:17:55.013 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964116995 2026-02-24T18:17:55.013 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 115964116995 2026-02-24T18:17:55.013 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 115964116995' 2026-02-24T18:17:55.013 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:17:55.238 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 1 -lt 115964116995 2026-02-24T18:17:55.239 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:17:56.240 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:17:56.240 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:17:56.459 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964116995 -lt 115964116995 2026-02-24T18:17:56.460 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:17:56.460 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-133143986179 2026-02-24T18:17:56.460 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:17:56.462 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:17:56.462 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-133143986179 2026-02-24T18:17:56.463 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:17:56.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=133143986179 2026-02-24T18:17:56.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 133143986179' 2026-02-24T18:17:56.465 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 133143986179 2026-02-24T18:17:56.465 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:17:56.678 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 133143986179 -lt 133143986179 2026-02-24T18:17:56.678 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:17:56.679 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-146028888067 2026-02-24T18:17:56.679 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:17:56.680 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:17:56.681 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-146028888067 2026-02-24T18:17:56.681 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:17:56.682 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=146028888067 2026-02-24T18:17:56.682 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 146028888067 2026-02-24T18:17:56.683 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 146028888067' 2026-02-24T18:17:56.683 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:17:56.911 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 146028888067 -lt 146028888067 2026-02-24T18:17:56.912 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:17:56.912 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-158913789955 2026-02-24T18:17:56.912 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:17:56.913 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T18:17:56.914 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-158913789955 2026-02-24T18:17:56.914 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:17:56.916 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=158913789955 2026-02-24T18:17:56.916 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 158913789955' 2026-02-24T18:17:56.916 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.3 seq 158913789955 2026-02-24T18:17:56.916 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T18:17:57.129 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 158913789955 -lt 158913789955 2026-02-24T18:17:57.129 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:17:57.130 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:17:57.130 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:17:57.407 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-24T18:17:57.407 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:17:57.407 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:17:57.407 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:17:57.407 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:17:57.407 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:17:57.407 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:17:57.407 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:17:57.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-24T18:17:57.628 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:17:57.628 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:17:57.628 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:17:57.911 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-24T18:17:57.912 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T18:17:57.912 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T18:17:57.912 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:22: run: teardown td/ec-error-rollforward 2026-02-24T18:17:57.912 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ec-error-rollforward 2026-02-24T18:17:57.912 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:17:57.912 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ec-error-rollforward KILL 2026-02-24T18:17:57.912 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:17:57.912 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:17:57.912 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:17:57.913 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:17:57.913 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:17:58.021 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:17:58.021 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:17:58.022 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:17:58.022 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:17:58.023 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:17:58.023 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:17:58.024 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:17:58.025 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:17:58.025 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:17:58.025 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:17:58.026 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:17:58.027 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:17:58.028 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:17:58.028 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ec-error-rollforward 2026-02-24T18:17:58.041 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:17:58.041 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:58.041 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:58.042 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.77100 2026-02-24T18:17:58.043 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:17:58.043 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:17:58.043 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2429: main: code=0 2026-02-24T18:17:58.043 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2433: main: teardown td/ec-error-rollforward 0 2026-02-24T18:17:58.043 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ec-error-rollforward 2026-02-24T18:17:58.043 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-24T18:17:58.043 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ec-error-rollforward KILL 2026-02-24T18:17:58.043 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:17:58.043 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:17:58.044 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:17:58.044 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:17:58.044 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:17:58.047 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:17:58.047 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:17:58.048 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:17:58.048 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:17:58.049 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:17:58.049 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:17:58.049 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:17:58.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:17:58.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:17:58.051 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:17:58.051 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:17:58.052 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:17:58.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-24T18:17:58.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ec-error-rollforward 2026-02-24T18:17:58.054 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:17:58.054 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:58.054 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.77100 2026-02-24T18:17:58.055 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.77100 2026-02-24T18:17:58.056 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:17:58.056 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:17:58.056 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2434: main: return 0 2026-02-24T18:17:58.057 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T18:17:58.057 DEBUG:teuthology.orchestra.run.trial172:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T18:17:58.107 INFO:tasks.workunit:Running workunit osd/osd-bench.sh... 2026-02-24T18:17:58.107 DEBUG:teuthology.orchestra.run.trial172:workunit test osd/osd-bench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=618c318a7a0b287ab5aeff207b7bf1d73ef45044 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh 2026-02-24T18:17:58.161 INFO:tasks.workunit.client.0.trial172.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-24T18:17:58.164 INFO:tasks.workunit.client.0.trial172.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-24T18:17:58.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T18:17:58.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T18:17:58.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: export PYTHONWARNINGS=ignore 2026-02-24T18:17:58.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: PYTHONWARNINGS=ignore 2026-02-24T18:17:58.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: export CEPH_CONF=/dev/null 2026-02-24T18:17:58.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: CEPH_CONF=/dev/null 2026-02-24T18:17:58.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: main: unset CEPH_ARGS 2026-02-24T18:17:58.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: main: local code 2026-02-24T18:17:58.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2428: main: run td/osd-bench 2026-02-24T18:17:58.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:22: run: local dir=td/osd-bench 2026-02-24T18:17:58.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:23: run: shift 2026-02-24T18:17:58.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:25: run: export CEPH_MON=127.0.0.1:7106 2026-02-24T18:17:58.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:25: run: CEPH_MON=127.0.0.1:7106 2026-02-24T18:17:58.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:26: run: export CEPH_ARGS 2026-02-24T18:17:58.165 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:27: run: uuidgen 2026-02-24T18:17:58.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:27: run: CEPH_ARGS+='--fsid=115a5927-34da-4ec2-8bc0-f4ff492167dd --auth-supported=none ' 2026-02-24T18:17:58.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7106 ' 2026-02-24T18:17:58.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:29: run: CEPH_ARGS+='--debug-bluestore 20 ' 2026-02-24T18:17:58.166 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:31: run: set 2026-02-24T18:17:58.166 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-24T18:17:58.168 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:31: run: local funcs=TEST_bench 2026-02-24T18:17:58.168 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:32: run: for func in $funcs 2026-02-24T18:17:58.168 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:33: run: setup td/osd-bench 2026-02-24T18:17:58.169 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-bench 2026-02-24T18:17:58.169 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-bench 2026-02-24T18:17:58.169 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bench 2026-02-24T18:17:58.169 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:17:58.169 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bench KILL 2026-02-24T18:17:58.169 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:17:58.169 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:17:58.169 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:17:58.169 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:17:58.169 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:17:58.170 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:17:58.170 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:17:58.171 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:17:58.171 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:17:58.172 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:17:58.173 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:17:58.173 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:17:58.174 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:17:58.174 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:17:58.175 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:17:58.175 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:17:58.176 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:17:58.177 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:17:58.177 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bench 2026-02-24T18:17:58.178 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:17:58.179 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:58.179 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.86286 2026-02-24T18:17:58.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.86286 2026-02-24T18:17:58.180 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:17:58.180 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:17:58.180 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-bench 2026-02-24T18:17:58.181 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T18:17:58.182 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:58.182 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.86286 2026-02-24T18:17:58.182 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.86286 2026-02-24T18:17:58.183 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T18:17:58.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-24T18:17:58.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-24T18:17:58.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T18:17:58.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-bench 1' TERM HUP INT 2026-02-24T18:17:58.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:34: run: TEST_bench td/osd-bench 2026-02-24T18:17:58.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:40: TEST_bench: local dir=td/osd-bench 2026-02-24T18:17:58.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:42: TEST_bench: run_mon td/osd-bench a 2026-02-24T18:17:58.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-bench 2026-02-24T18:17:58.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T18:17:58.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T18:17:58.185 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T18:17:58.186 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-bench/a 2026-02-24T18:17:58.186 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bench/a --run-dir=td/osd-bench 2026-02-24T18:17:58.209 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T18:17:58.209 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:17:58.209 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:17:58.210 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:17:58.210 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:58.210 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.86286 2026-02-24T18:17:58.210 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.86286/$cluster-$name.asok' 2026-02-24T18:17:58.210 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bench/a '--log-file=td/osd-bench/$name.log' '--admin-socket=/tmp/ceph-asok.86286/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bench/log --run-dir=td/osd-bench '--pid-file=td/osd-bench/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T18:17:58.238 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T18:17:58.239 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T18:17:58.239 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:17:58.239 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:17:58.239 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T18:17:58.239 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T18:17:58.241 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:17:58.241 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:17:58.241 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:17:58.241 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:17:58.241 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:58.241 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.86286 2026-02-24T18:17:58.241 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.86286/ceph-mon.a.asok 2026-02-24T18:17:58.241 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:17:58.241 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.86286/ceph-mon.a.asok config get fsid 2026-02-24T18:17:58.296 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T18:17:58.296 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:17:58.297 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:17:58.297 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T18:17:58.297 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T18:17:58.297 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:17:58.297 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:17:58.297 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:17:58.297 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:17:58.297 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:58.297 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.86286 2026-02-24T18:17:58.297 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.86286/ceph-mon.a.asok 2026-02-24T18:17:58.298 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:17:58.298 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.86286/ceph-mon.a.asok config get mon_host 2026-02-24T18:17:58.350 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:43: TEST_bench: run_mgr td/osd-bench x 2026-02-24T18:17:58.350 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-bench 2026-02-24T18:17:58.350 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T18:17:58.350 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T18:17:58.351 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T18:17:58.351 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-bench/x 2026-02-24T18:17:58.351 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T18:17:58.464 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T18:17:58.464 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:17:58.464 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:17:58.465 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:17:58.465 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:58.465 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.86286 2026-02-24T18:17:58.465 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.86286/$cluster-$name.asok' 2026-02-24T18:17:58.465 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:17:58.467 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bench/x '--log-file=td/osd-bench/$name.log' '--admin-socket=/tmp/ceph-asok.86286/$cluster-$name.asok' --run-dir=td/osd-bench '--pid-file=td/osd-bench/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:44: TEST_bench: run_osd td/osd-bench 0 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-bench 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-bench/0 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=115a5927-34da-4ec2-8bc0-f4ff492167dd --auth-supported=none --mon-host=127.0.0.1:7106 --debug-bluestore 20 ' 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-bench/0' 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-bench/0/journal' 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-bench' 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:17:58.484 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.86286 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.86286/$cluster-$name.asok' 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86286/$cluster-$name.asok' 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-bench/$name.log' 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-bench/$name.pid' 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:17:58.485 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-bench/0 2026-02-24T18:17:58.486 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:17:58.487 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a596d3f8-4db0-48ee-ac0d-3a46c0083beb 2026-02-24T18:17:58.487 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 a596d3f8-4db0-48ee-ac0d-3a46c0083beb' 2026-02-24T18:17:58.487 INFO:tasks.workunit.client.0.trial172.stdout:add osd0 a596d3f8-4db0-48ee-ac0d-3a46c0083beb 2026-02-24T18:17:58.488 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:17:58.498 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBW651p4ICIHRAAA8IUTnupJRvC5/iLwm0jnA== 2026-02-24T18:17:58.498 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBW651p4ICIHRAAA8IUTnupJRvC5/iLwm0jnA=="}' 2026-02-24T18:17:58.498 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a596d3f8-4db0-48ee-ac0d-3a46c0083beb -i td/osd-bench/0/new.json 2026-02-24T18:17:58.614 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:17:58.622 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-bench/0/new.json 2026-02-24T18:17:58.623 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=115a5927-34da-4ec2-8bc0-f4ff492167dd --auth-supported=none --mon-host=127.0.0.1:7106 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bench/0 --osd-journal=td/osd-bench/0/journal --chdir= --run-dir=td/osd-bench '--admin-socket=/tmp/ceph-asok.86286/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bench/$name.log' '--pid-file=td/osd-bench/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBW651p4ICIHRAAA8IUTnupJRvC5/iLwm0jnA== --osd-uuid a596d3f8-4db0-48ee-ac0d-3a46c0083beb 2026-02-24T18:17:58.638 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:58.632+0000 7fefb9540a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:58.639 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:58.636+0000 7fefb9540a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:58.640 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:58.636+0000 7fefb9540a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:58.641 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:58.636+0000 7fefb9540a40 -1 bdev(0x562ce6890400 td/osd-bench/0/block) open stat got: (1) Operation not permitted 2026-02-24T18:17:58.641 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:58.636+0000 7fefb9540a40 -1 bluestore(td/osd-bench/0) _read_fsid unparsable uuid 2026-02-24T18:17:59.387 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-bench/0/keyring 2026-02-24T18:17:59.387 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:17:59.387 INFO:tasks.workunit.client.0.trial172.stdout:adding osd0 key to auth repository 2026-02-24T18:17:59.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T18:17:59.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-bench/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:17:59.508 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:17:59.508 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T18:17:59.508 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=115a5927-34da-4ec2-8bc0-f4ff492167dd --auth-supported=none --mon-host=127.0.0.1:7106 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bench/0 --osd-journal=td/osd-bench/0/journal --chdir= --run-dir=td/osd-bench '--admin-socket=/tmp/ceph-asok.86286/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bench/$name.log' '--pid-file=td/osd-bench/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:17:59.509 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:17:59.509 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:17:59.509 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:17:59.531 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:59.528+0000 7f86b845ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:59.533 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:59.528+0000 7f86b845ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:59.535 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:59.528+0000 7f86b845ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:17:59.620 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T18:17:59.620 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:17:59.621 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:17:59.621 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:17:59.621 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:17:59.621 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:17:59.621 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:17:59.622 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:17:59.622 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:17:59.622 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:17:59.727 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:17:59.925 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:17:59.920+0000 7f86b845ea40 -1 Falling back to public interface 2026-02-24T18:18:00.728 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:18:00.729 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:18:00.729 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:18:00.729 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:18:00.730 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:18:00.730 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:18:00.840 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:18:00.901 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:00.896+0000 7f86b845ea40 -1 osd.0 0 log_to_monitors true 2026-02-24T18:18:01.841 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:18:01.841 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:18:01.842 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:18:01.842 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:18:01.842 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:18:01.842 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:18:02.050 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/466043459,v1:127.0.0.1:6801/466043459] [v2:127.0.0.1:6802/466043459,v1:127.0.0.1:6803/466043459] exists,up a596d3f8-4db0-48ee-ac0d-3a46c0083beb 2026-02-24T18:18:02.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:18:02.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:18:02.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:18:02.050 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:47: TEST_bench: CEPH_ARGS= 2026-02-24T18:18:02.050 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:47: TEST_bench: ceph-conf --show-config-value osd_bench_small_size_max_iops 2026-02-24T18:18:02.059 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:47: TEST_bench: local osd_bench_small_size_max_iops=100 2026-02-24T18:18:02.060 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:49: TEST_bench: CEPH_ARGS= 2026-02-24T18:18:02.060 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:49: TEST_bench: ceph-conf --show-config-value osd_bench_large_size_max_throughput 2026-02-24T18:18:02.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:49: TEST_bench: local osd_bench_large_size_max_throughput=104857600 2026-02-24T18:18:02.070 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:51: TEST_bench: CEPH_ARGS= 2026-02-24T18:18:02.070 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:51: TEST_bench: ceph-conf --show-config-value osd_bench_max_block_size 2026-02-24T18:18:02.084 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:51: TEST_bench: local osd_bench_max_block_size=67108864 2026-02-24T18:18:02.084 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:53: TEST_bench: CEPH_ARGS= 2026-02-24T18:18:02.084 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:53: TEST_bench: ceph-conf --show-config-value osd_bench_duration 2026-02-24T18:18:02.094 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:53: TEST_bench: local osd_bench_duration=30 2026-02-24T18:18:02.094 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:58: TEST_bench: expect_failure td/osd-bench osd_bench_max_block_size ceph tell osd.0 bench 1024 67108865 2026-02-24T18:18:02.094 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2045: expect_failure: local dir=td/osd-bench 2026-02-24T18:18:02.094 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2046: expect_failure: shift 2026-02-24T18:18:02.094 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2047: expect_failure: local expected=osd_bench_max_block_size 2026-02-24T18:18:02.094 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2048: expect_failure: shift 2026-02-24T18:18:02.094 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2049: expect_failure: local success 2026-02-24T18:18:02.094 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2051: expect_failure: ceph tell osd.0 bench 1024 67108865 2026-02-24T18:18:02.177 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2054: expect_failure: success=false 2026-02-24T18:18:02.177 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2057: expect_failure: false 2026-02-24T18:18:02.177 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2057: expect_failure: grep --quiet osd_bench_max_block_size td/osd-bench/out 2026-02-24T18:18:02.178 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2061: expect_failure: return 0 2026-02-24T18:18:02.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:64: TEST_bench: local bsize=1024 2026-02-24T18:18:02.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:65: TEST_bench: local max_count=3072000 2026-02-24T18:18:02.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:66: TEST_bench: expect_failure td/osd-bench bench_small_size_max_iops ceph tell osd.0 bench 3072001 1024 2026-02-24T18:18:02.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2045: expect_failure: local dir=td/osd-bench 2026-02-24T18:18:02.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2046: expect_failure: shift 2026-02-24T18:18:02.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2047: expect_failure: local expected=bench_small_size_max_iops 2026-02-24T18:18:02.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2048: expect_failure: shift 2026-02-24T18:18:02.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2049: expect_failure: local success 2026-02-24T18:18:02.179 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2051: expect_failure: ceph tell osd.0 bench 3072001 1024 2026-02-24T18:18:02.249 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2054: expect_failure: success=false 2026-02-24T18:18:02.249 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2057: expect_failure: false 2026-02-24T18:18:02.249 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2057: expect_failure: grep --quiet bench_small_size_max_iops td/osd-bench/out 2026-02-24T18:18:02.251 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2061: expect_failure: return 0 2026-02-24T18:18:02.251 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:72: TEST_bench: local bsize=1048577 2026-02-24T18:18:02.251 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:73: TEST_bench: local max_count=3145728000 2026-02-24T18:18:02.251 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:74: TEST_bench: expect_failure td/osd-bench osd_bench_large_size_max_throughput ceph tell osd.0 bench 3145728001 1048577 2026-02-24T18:18:02.251 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2045: expect_failure: local dir=td/osd-bench 2026-02-24T18:18:02.251 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2046: expect_failure: shift 2026-02-24T18:18:02.251 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2047: expect_failure: local expected=osd_bench_large_size_max_throughput 2026-02-24T18:18:02.251 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2048: expect_failure: shift 2026-02-24T18:18:02.252 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2049: expect_failure: local success 2026-02-24T18:18:02.252 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2051: expect_failure: ceph tell osd.0 bench 3145728001 1048577 2026-02-24T18:18:02.313 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2054: expect_failure: success=false 2026-02-24T18:18:02.313 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2057: expect_failure: false 2026-02-24T18:18:02.313 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2057: expect_failure: grep --quiet osd_bench_large_size_max_throughput td/osd-bench/out 2026-02-24T18:18:02.314 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2061: expect_failure: return 0 2026-02-24T18:18:02.314 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:80: TEST_bench: ceph tell osd.0 bench 2026-02-24T18:18:03.970 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:18:03.970 INFO:tasks.workunit.client.0.trial172.stdout: "bytes_written": 1073741824, 2026-02-24T18:18:03.970 INFO:tasks.workunit.client.0.trial172.stdout: "blocksize": 4194304, 2026-02-24T18:18:03.970 INFO:tasks.workunit.client.0.trial172.stdout: "elapsed_sec": 0.70524354600000005, 2026-02-24T18:18:03.970 INFO:tasks.workunit.client.0.trial172.stdout: "bytes_per_sec": 1522512088.327569, 2026-02-24T18:18:03.970 INFO:tasks.workunit.client.0.trial172.stdout: "iops": 362.99516876401162 2026-02-24T18:18:03.970 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:18:03.985 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:84: TEST_bench: ceph tell osd.0 bench 10 14456 4444 3 2026-02-24T18:18:04.056 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:18:04.056 INFO:tasks.workunit.client.0.trial172.stdout: "bytes_written": 10, 2026-02-24T18:18:04.056 INFO:tasks.workunit.client.0.trial172.stdout: "blocksize": 14456, 2026-02-24T18:18:04.057 INFO:tasks.workunit.client.0.trial172.stdout: "elapsed_sec": 0.00031352099999999998, 2026-02-24T18:18:04.057 INFO:tasks.workunit.client.0.trial172.stdout: "bytes_per_sec": 31895.790074668046, 2026-02-24T18:18:04.057 INFO:tasks.workunit.client.0.trial172.stdout: "iops": 2.2064049581259026 2026-02-24T18:18:04.057 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:18:04.063 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:90: TEST_bench: ceph tell osd.0 bench 1 14456 2026-02-24T18:18:04.138 INFO:tasks.workunit.client.0.trial172.stdout:{ 2026-02-24T18:18:04.138 INFO:tasks.workunit.client.0.trial172.stdout: "bytes_written": 1, 2026-02-24T18:18:04.139 INFO:tasks.workunit.client.0.trial172.stdout: "blocksize": 14456, 2026-02-24T18:18:04.139 INFO:tasks.workunit.client.0.trial172.stdout: "elapsed_sec": 0.000197486, 2026-02-24T18:18:04.139 INFO:tasks.workunit.client.0.trial172.stdout: "bytes_per_sec": 5063.6500815247664, 2026-02-24T18:18:04.139 INFO:tasks.workunit.client.0.trial172.stdout: "iops": 0.3502801661265057 2026-02-24T18:18:04.139 INFO:tasks.workunit.client.0.trial172.stdout:} 2026-02-24T18:18:04.149 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:35: run: teardown td/osd-bench 2026-02-24T18:18:04.149 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bench 2026-02-24T18:18:04.149 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:18:04.149 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bench KILL 2026-02-24T18:18:04.150 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:18:04.150 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:18:04.150 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:18:04.150 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:18:04.150 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:18:04.258 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:18:04.258 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:18:04.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:18:04.259 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:18:04.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:18:04.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:18:04.261 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:18:04.262 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:18:04.262 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:18:04.262 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:18:04.262 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:18:04.263 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:18:04.264 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:18:04.264 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bench 2026-02-24T18:18:04.298 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:18:04.298 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:18:04.299 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.86286 2026-02-24T18:18:04.299 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.86286 2026-02-24T18:18:04.300 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:18:04.301 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:18:04.301 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2429: main: code=0 2026-02-24T18:18:04.301 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2433: main: teardown td/osd-bench 0 2026-02-24T18:18:04.301 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bench 2026-02-24T18:18:04.301 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-24T18:18:04.301 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bench KILL 2026-02-24T18:18:04.301 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:18:04.301 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:18:04.301 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:18:04.301 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:18:04.301 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:18:04.302 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:18:04.303 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:18:04.304 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:18:04.304 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:18:04.305 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:18:04.305 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:18:04.305 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:18:04.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:18:04.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:18:04.307 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:18:04.307 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:18:04.308 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:18:04.309 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-24T18:18:04.309 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bench 2026-02-24T18:18:04.310 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:18:04.310 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:18:04.310 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.86286 2026-02-24T18:18:04.310 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.86286 2026-02-24T18:18:04.311 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:18:04.312 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:18:04.312 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2434: main: return 0 2026-02-24T18:18:04.313 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T18:18:04.313 DEBUG:teuthology.orchestra.run.trial172:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T18:18:04.368 INFO:tasks.workunit:Running workunit osd/osd-bluefs-volume-ops.sh... 2026-02-24T18:18:04.368 DEBUG:teuthology.orchestra.run.trial172:workunit test osd/osd-bluefs-volume-ops.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=618c318a7a0b287ab5aeff207b7bf1d73ef45044 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh 2026-02-24T18:18:04.421 INFO:tasks.workunit.client.0.trial172.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-24T18:18:04.426 INFO:tasks.workunit.client.0.trial172.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-24T18:18:04.426 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T18:18:04.426 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T18:18:04.426 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: export PYTHONWARNINGS=ignore 2026-02-24T18:18:04.426 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: PYTHONWARNINGS=ignore 2026-02-24T18:18:04.426 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: export CEPH_CONF=/dev/null 2026-02-24T18:18:04.427 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: CEPH_CONF=/dev/null 2026-02-24T18:18:04.427 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: main: unset CEPH_ARGS 2026-02-24T18:18:04.427 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: main: local code 2026-02-24T18:18:04.427 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2428: main: run td/osd-bluefs-volume-ops 2026-02-24T18:18:04.427 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:8: run: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:18:04.427 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:9: run: shift 2026-02-24T18:18:04.427 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:11: run: set 2026-02-24T18:18:04.427 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:11: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-24T18:18:04.429 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:11: run: local 'funcs=TEST_bluestore 2026-02-24T18:18:04.429 INFO:tasks.workunit.client.0.trial172.stderr:TEST_bluestore2 2026-02-24T18:18:04.429 INFO:tasks.workunit.client.0.trial172.stderr:TEST_bluestore_expand' 2026-02-24T18:18:04.429 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:12: run: for func in $funcs 2026-02-24T18:18:04.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:13: run: setup td/osd-bluefs-volume-ops 2026-02-24T18:18:04.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:18:04.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-bluefs-volume-ops 2026-02-24T18:18:04.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:18:04.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:18:04.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-24T18:18:04.430 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:18:04.430 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:18:04.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:18:04.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:18:04.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:18:04.431 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:18:04.431 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:18:04.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:18:04.432 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:18:04.433 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:18:04.433 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:18:04.434 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:18:04.435 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:18:04.435 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:18:04.435 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:18:04.435 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:18:04.436 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:18:04.438 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:18:04.438 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-24T18:18:04.439 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:18:04.439 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:18:04.439 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:18:04.440 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.88188 2026-02-24T18:18:04.441 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:18:04.441 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:18:04.441 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-bluefs-volume-ops 2026-02-24T18:18:04.442 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T18:18:04.443 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:18:04.443 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:18:04.443 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.88188 2026-02-24T18:18:04.444 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T18:18:04.445 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-24T18:18:04.445 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-24T18:18:04.445 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T18:18:04.445 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-bluefs-volume-ops 1' TERM HUP INT 2026-02-24T18:18:04.446 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: TEST_bluestore td/osd-bluefs-volume-ops 2026-02-24T18:18:04.446 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:20: TEST_bluestore: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:18:04.446 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:22: TEST_bluestore: ulimit -n 2026-02-24T18:18:04.447 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:22: TEST_bluestore: local flimit=4096 2026-02-24T18:18:04.447 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:23: TEST_bluestore: '[' 4096 -lt 1536 ']' 2026-02-24T18:18:04.447 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:26: TEST_bluestore: export CEPH_MON=127.0.0.1:7146 2026-02-24T18:18:04.447 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:26: TEST_bluestore: CEPH_MON=127.0.0.1:7146 2026-02-24T18:18:04.447 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:27: TEST_bluestore: export CEPH_ARGS 2026-02-24T18:18:04.447 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:28: TEST_bluestore: uuidgen 2026-02-24T18:18:04.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:28: TEST_bluestore: CEPH_ARGS+='--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none ' 2026-02-24T18:18:04.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:29: TEST_bluestore: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-24T18:18:04.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:30: TEST_bluestore: CEPH_ARGS+='--bluestore_block_size=2147483648 ' 2026-02-24T18:18:04.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:31: TEST_bluestore: CEPH_ARGS+='--bluestore_block_db_create=true ' 2026-02-24T18:18:04.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:32: TEST_bluestore: CEPH_ARGS+='--bluestore_block_db_size=1073741824 ' 2026-02-24T18:18:04.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:33: TEST_bluestore: CEPH_ARGS+='--bluestore_block_wal_size=536870912 ' 2026-02-24T18:18:04.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:34: TEST_bluestore: CEPH_ARGS+='--bluestore_block_wal_create=true ' 2026-02-24T18:18:04.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:35: TEST_bluestore: CEPH_ARGS+='--bluestore_fsck_on_mount=true ' 2026-02-24T18:18:04.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:37: TEST_bluestore: CEPH_ARGS+='--bluestore_allocation_from_file=0 ' 2026-02-24T18:18:04.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:39: TEST_bluestore: run_mon td/osd-bluefs-volume-ops a 2026-02-24T18:18:04.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:18:04.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T18:18:04.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T18:18:04.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T18:18:04.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-bluefs-volume-ops/a 2026-02-24T18:18:04.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bluefs-volume-ops/a --run-dir=td/osd-bluefs-volume-ops 2026-02-24T18:18:04.469 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T18:18:04.469 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:18:04.470 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:18:04.470 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:18:04.470 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:18:04.470 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:18:04.470 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:18:04.470 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bluefs-volume-ops/a '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bluefs-volume-ops/log --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T18:18:04.498 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T18:18:04.499 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T18:18:04.499 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:18:04.499 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:18:04.499 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T18:18:04.500 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T18:18:04.500 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:18:04.501 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:18:04.501 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:18:04.501 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:18:04.501 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:18:04.501 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:18:04.501 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.88188/ceph-mon.a.asok 2026-02-24T18:18:04.501 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:18:04.501 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.88188/ceph-mon.a.asok config get fsid 2026-02-24T18:18:04.557 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T18:18:04.558 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:18:04.558 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:18:04.558 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T18:18:04.558 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T18:18:04.558 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:18:04.558 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:18:04.558 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:18:04.558 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:18:04.558 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:18:04.558 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:18:04.558 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.88188/ceph-mon.a.asok 2026-02-24T18:18:04.559 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:18:04.559 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.88188/ceph-mon.a.asok config get mon_host 2026-02-24T18:18:04.611 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:40: TEST_bluestore: run_mgr td/osd-bluefs-volume-ops x 2026-02-24T18:18:04.611 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:18:04.611 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T18:18:04.611 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T18:18:04.611 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T18:18:04.611 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-bluefs-volume-ops/x 2026-02-24T18:18:04.611 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T18:18:04.724 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T18:18:04.725 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:18:04.725 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:18:04.725 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:18:04.725 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:18:04.725 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:18:04.725 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:18:04.726 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:18:04.726 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bluefs-volume-ops/x '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:18:04.748 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:41: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 0 2026-02-24T18:18:04.748 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:18:04.748 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:18:04.748 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T18:18:04.748 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:18:04.749 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-24T18:18:04.749 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:18:04.749 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:18:04.749 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:18:04.749 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:18:04.749 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-24T18:18:04.749 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-24T18:18:04.749 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:18:04.749 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:18:04.750 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:18:04.750 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:18:04.750 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:18:04.750 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:18:04.750 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:18:04.750 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:18:04.750 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:18:04.750 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:18:04.750 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:18:04.750 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:18:04.750 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:18:04.751 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:18:04.751 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:18:04.751 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:18:04.751 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:18:04.751 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:18:04.751 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:18:04.751 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:18:04.751 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:18:04.751 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:18:04.751 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-24T18:18:04.751 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:18:04.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=410d8f94-9514-413c-9295-e8e271a743d8 2026-02-24T18:18:04.752 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 410d8f94-9514-413c-9295-e8e271a743d8' 2026-02-24T18:18:04.753 INFO:tasks.workunit.client.0.trial172.stdout:add osd0 410d8f94-9514-413c-9295-e8e271a743d8 2026-02-24T18:18:04.753 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:18:04.760 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBc651pWWAqLRAAPkEr26nNP6seIt15WQN25A== 2026-02-24T18:18:04.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBc651pWWAqLRAAPkEr26nNP6seIt15WQN25A=="}' 2026-02-24T18:18:04.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 410d8f94-9514-413c-9295-e8e271a743d8 -i td/osd-bluefs-volume-ops/0/new.json 2026-02-24T18:18:04.874 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:18:04.882 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-bluefs-volume-ops/0/new.json 2026-02-24T18:18:04.883 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBc651pWWAqLRAAPkEr26nNP6seIt15WQN25A== --osd-uuid 410d8f94-9514-413c-9295-e8e271a743d8 2026-02-24T18:18:04.900 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:04.896+0000 7fc2dda85a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:04.901 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:04.896+0000 7fc2dda85a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:04.902 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:04.896+0000 7fc2dda85a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:04.903 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:04.896+0000 7fc2dda85a40 -1 bdev(0x558b9aae6400 td/osd-bluefs-volume-ops/0/block) open stat got: (1) Operation not permitted 2026-02-24T18:18:04.903 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:04.896+0000 7fc2dda85a40 -1 bluestore(td/osd-bluefs-volume-ops/0) _read_fsid unparsable uuid 2026-02-24T18:18:06.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-bluefs-volume-ops/0/keyring 2026-02-24T18:18:06.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:18:06.889 INFO:tasks.workunit.client.0.trial172.stdout:adding osd0 key to auth repository 2026-02-24T18:18:06.889 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T18:18:06.889 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-bluefs-volume-ops/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:18:07.051 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:18:07.052 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T18:18:07.052 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:18:07.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:18:07.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:18:07.053 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:18:07.066 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:07.060+0000 7f9025905a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:07.068 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:07.064+0000 7f9025905a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:07.069 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:07.064+0000 7f9025905a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:07.162 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T18:18:07.162 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:18:07.162 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:18:07.162 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:18:07.162 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:18:07.162 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:18:07.162 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:18:07.163 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:18:07.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:18:07.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:18:07.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:18:07.629 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:07.624+0000 7f9025905a40 -1 Falling back to public interface 2026-02-24T18:18:08.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:18:08.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:18:08.375 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:18:08.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:18:08.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:18:08.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:18:08.584 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:18:08.838 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:08.832+0000 7f9025905a40 -1 osd.0 0 log_to_monitors true 2026-02-24T18:18:09.585 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:18:09.585 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:18:09.585 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:18:09.585 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:18:09.586 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:18:09.586 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:18:09.795 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2741173553,v1:127.0.0.1:6803/2741173553] [v2:127.0.0.1:6804/2741173553,v1:127.0.0.1:6805/2741173553] exists,up 410d8f94-9514-413c-9295-e8e271a743d8 2026-02-24T18:18:09.795 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:18:09.796 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:18:09.796 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:18:09.796 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:42: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-24T18:18:09.799 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:42: TEST_bluestore: osd_pid0=89666 2026-02-24T18:18:09.799 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:43: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 1 2026-02-24T18:18:09.799 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:18:09.799 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:18:09.800 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T18:18:09.800 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:18:09.800 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-24T18:18:09.800 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:18:09.800 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:18:09.800 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:18:09.800 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:18:09.800 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-24T18:18:09.800 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-24T18:18:09.800 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:18:09.801 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:18:09.801 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:18:09.801 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:18:09.801 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:18:09.801 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:18:09.801 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:18:09.801 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:18:09.801 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:18:09.801 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:18:09.801 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:18:09.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:18:09.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:18:09.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:18:09.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:18:09.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:18:09.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:18:09.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:18:09.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:18:09.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:18:09.803 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:18:09.803 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:18:09.803 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-24T18:18:09.803 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:18:09.804 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f4b079dc-13d9-4358-9502-d9202bc6b891 2026-02-24T18:18:09.804 INFO:tasks.workunit.client.0.trial172.stdout:add osd1 f4b079dc-13d9-4358-9502-d9202bc6b891 2026-02-24T18:18:09.805 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 f4b079dc-13d9-4358-9502-d9202bc6b891' 2026-02-24T18:18:09.805 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:18:09.813 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBh651pyVdMMBAA4a0/xCNewY6jyIMP0EWPDg== 2026-02-24T18:18:09.813 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBh651pyVdMMBAA4a0/xCNewY6jyIMP0EWPDg=="}' 2026-02-24T18:18:09.813 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f4b079dc-13d9-4358-9502-d9202bc6b891 -i td/osd-bluefs-volume-ops/1/new.json 2026-02-24T18:18:10.035 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:18:10.046 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-bluefs-volume-ops/1/new.json 2026-02-24T18:18:10.047 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBh651pyVdMMBAA4a0/xCNewY6jyIMP0EWPDg== --osd-uuid f4b079dc-13d9-4358-9502-d9202bc6b891 2026-02-24T18:18:10.063 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:10.056+0000 7f0ba2a53a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:10.064 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:10.060+0000 7f0ba2a53a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:10.065 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:10.060+0000 7f0ba2a53a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:10.066 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:10.060+0000 7f0ba2a53a40 -1 bdev(0x55913b97c400 td/osd-bluefs-volume-ops/1/block) open stat got: (1) Operation not permitted 2026-02-24T18:18:10.066 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:10.060+0000 7f0ba2a53a40 -1 bluestore(td/osd-bluefs-volume-ops/1) _read_fsid unparsable uuid 2026-02-24T18:18:11.486 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-bluefs-volume-ops/1/keyring 2026-02-24T18:18:11.486 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:18:11.487 INFO:tasks.workunit.client.0.trial172.stdout:adding osd1 key to auth repository 2026-02-24T18:18:11.487 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T18:18:11.487 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-bluefs-volume-ops/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:18:11.776 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:18:11.778 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T18:18:11.778 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:18:11.778 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:18:11.778 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:18:11.778 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:18:11.793 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:11.788+0000 7f37bd82da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:11.794 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:11.788+0000 7f37bd82da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:11.795 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:11.788+0000 7f37bd82da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:11.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T18:18:11.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:18:11.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:18:11.984 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:18:11.984 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:18:11.984 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:18:11.984 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:18:11.984 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:18:11.984 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:18:11.984 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:18:12.205 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:18:12.305 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:12.300+0000 7f37bd82da40 -1 Falling back to public interface 2026-02-24T18:18:13.207 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:18:13.207 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:18:13.208 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:18:13.208 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:18:13.209 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:18:13.209 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:18:13.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:18:13.559 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:13.552+0000 7f37bd82da40 -1 osd.1 0 log_to_monitors true 2026-02-24T18:18:14.425 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:18:14.425 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:18:14.425 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:18:14.425 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:18:14.425 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:18:14.425 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:18:14.636 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3337510542,v1:127.0.0.1:6811/3337510542] [v2:127.0.0.1:6812/3337510542,v1:127.0.0.1:6813/3337510542] exists,up f4b079dc-13d9-4358-9502-d9202bc6b891 2026-02-24T18:18:14.636 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:18:14.636 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:18:14.636 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:18:14.637 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:44: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:44: TEST_bluestore: osd_pid1=92121 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:45: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 2 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:18:14.639 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:18:14.640 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:18:14.640 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:18:14.640 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:18:14.640 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:18:14.640 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:18:14.640 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:18:14.640 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:18:14.640 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:18:14.640 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:18:14.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:18:14.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:18:14.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:18:14.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:18:14.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:18:14.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:18:14.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:18:14.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-24T18:18:14.641 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:18:14.642 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c77df6a7-2ace-45f0-9cdb-0238c4cea4fe 2026-02-24T18:18:14.643 INFO:tasks.workunit.client.0.trial172.stdout:add osd2 c77df6a7-2ace-45f0-9cdb-0238c4cea4fe 2026-02-24T18:18:14.643 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 c77df6a7-2ace-45f0-9cdb-0238c4cea4fe' 2026-02-24T18:18:14.643 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:18:14.654 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBm651p3yPUJhAAW7gRd800Iz3QM94dLYUmJQ== 2026-02-24T18:18:14.654 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBm651p3yPUJhAAW7gRd800Iz3QM94dLYUmJQ=="}' 2026-02-24T18:18:14.654 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c77df6a7-2ace-45f0-9cdb-0238c4cea4fe -i td/osd-bluefs-volume-ops/2/new.json 2026-02-24T18:18:14.873 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:18:14.882 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-bluefs-volume-ops/2/new.json 2026-02-24T18:18:14.883 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBm651p3yPUJhAAW7gRd800Iz3QM94dLYUmJQ== --osd-uuid c77df6a7-2ace-45f0-9cdb-0238c4cea4fe 2026-02-24T18:18:14.899 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:14.892+0000 7ffac88bba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:14.900 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:14.896+0000 7ffac88bba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:14.901 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:14.896+0000 7ffac88bba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:14.902 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:14.896+0000 7ffac88bba40 -1 bdev(0x55e9ab748400 td/osd-bluefs-volume-ops/2/block) open stat got: (1) Operation not permitted 2026-02-24T18:18:14.902 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:14.896+0000 7ffac88bba40 -1 bluestore(td/osd-bluefs-volume-ops/2) _read_fsid unparsable uuid 2026-02-24T18:18:16.442 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-bluefs-volume-ops/2/keyring 2026-02-24T18:18:16.442 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:18:16.443 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T18:18:16.443 INFO:tasks.workunit.client.0.trial172.stdout:adding osd2 key to auth repository 2026-02-24T18:18:16.443 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-bluefs-volume-ops/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:18:16.744 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:18:16.744 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T18:18:16.745 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:18:16.745 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:18:16.745 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:18:16.745 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:18:16.761 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:16.756+0000 7f5b8108aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:16.762 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:16.756+0000 7f5b8108aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:16.763 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:16.760+0000 7f5b8108aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:16.962 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T18:18:16.962 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:18:16.962 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:18:16.962 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:18:16.963 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:18:16.963 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:18:16.963 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:18:16.963 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:18:16.964 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:18:16.964 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:18:17.171 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:18:17.245 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:17.240+0000 7f5b8108aa40 -1 Falling back to public interface 2026-02-24T18:18:18.173 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:18:18.173 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:18:18.173 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:18:18.174 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:18:18.174 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:18:18.174 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:18:18.386 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:18:18.523 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:18.519+0000 7f5b8108aa40 -1 osd.2 0 log_to_monitors true 2026-02-24T18:18:19.387 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:18:19.387 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:18:19.387 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:18:19.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:18:19.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:18:19.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:18:19.600 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2273843503,v1:127.0.0.1:6819/2273843503] [v2:127.0.0.1:6820/2273843503,v1:127.0.0.1:6821/2273843503] exists,up c77df6a7-2ace-45f0-9cdb-0238c4cea4fe 2026-02-24T18:18:19.600 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:18:19.601 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:18:19.601 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:18:19.601 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:46: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-24T18:18:19.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:46: TEST_bluestore: osd_pid2=94268 2026-02-24T18:18:19.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:47: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 3 2026-02-24T18:18:19.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:18:19.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:18:19.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T18:18:19.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:18:19.602 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-24T18:18:19.603 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:18:19.603 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:18:19.603 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:18:19.603 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:18:19.603 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-24T18:18:19.603 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-24T18:18:19.603 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:18:19.603 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:18:19.603 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:18:19.603 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:18:19.603 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:18:19.603 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:18:19.604 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:18:19.604 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:18:19.604 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:18:19.604 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:18:19.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:18:19.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:18:19.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:18:19.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:18:19.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:18:19.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:18:19.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:18:19.605 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:18:19.605 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:18:19.605 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:18:19.605 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:18:19.605 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:18:19.605 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-24T18:18:19.605 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:18:19.606 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=79d5a161-1bb7-472e-8989-c86f625396ef 2026-02-24T18:18:19.606 INFO:tasks.workunit.client.0.trial172.stdout:add osd3 79d5a161-1bb7-472e-8989-c86f625396ef 2026-02-24T18:18:19.606 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 79d5a161-1bb7-472e-8989-c86f625396ef' 2026-02-24T18:18:19.606 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:18:19.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBr651pIvyuJBAAKyGHgjPAB+5qBaFaa5hkHQ== 2026-02-24T18:18:19.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBr651pIvyuJBAAKyGHgjPAB+5qBaFaa5hkHQ=="}' 2026-02-24T18:18:19.618 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 79d5a161-1bb7-472e-8989-c86f625396ef -i td/osd-bluefs-volume-ops/3/new.json 2026-02-24T18:18:19.842 INFO:tasks.workunit.client.0.trial172.stdout:3 2026-02-24T18:18:19.852 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-bluefs-volume-ops/3/new.json 2026-02-24T18:18:19.854 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBr651pIvyuJBAAKyGHgjPAB+5qBaFaa5hkHQ== --osd-uuid 79d5a161-1bb7-472e-8989-c86f625396ef 2026-02-24T18:18:19.878 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:19.871+0000 7fd142fada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:19.879 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:19.875+0000 7fd142fada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:19.881 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:19.875+0000 7fd142fada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:19.881 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:19.875+0000 7fd142fada40 -1 bdev(0x5617ff66a400 td/osd-bluefs-volume-ops/3/block) open stat got: (1) Operation not permitted 2026-02-24T18:18:19.882 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:19.875+0000 7fd142fada40 -1 bluestore(td/osd-bluefs-volume-ops/3) _read_fsid unparsable uuid 2026-02-24T18:18:21.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-bluefs-volume-ops/3/keyring 2026-02-24T18:18:21.307 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:18:21.308 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T18:18:21.308 INFO:tasks.workunit.client.0.trial172.stdout:adding osd3 key to auth repository 2026-02-24T18:18:21.308 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-bluefs-volume-ops/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:18:21.597 INFO:tasks.workunit.client.0.trial172.stdout:start osd.3 2026-02-24T18:18:21.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T18:18:21.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:18:21.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:18:21.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:18:21.598 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:18:21.620 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:21.615+0000 7fd40ffe0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:21.622 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:21.615+0000 7fd40ffe0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:21.623 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:21.619+0000 7fd40ffe0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:18:21.817 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T18:18:21.818 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:18:21.818 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T18:18:21.818 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:18:21.818 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:18:21.818 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:18:21.818 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:18:21.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:18:21.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:18:21.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:18:22.037 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:18:22.092 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:22.087+0000 7fd40ffe0a40 -1 Falling back to public interface 2026-02-24T18:18:23.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:18:23.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:18:23.039 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:18:23.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:18:23.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:18:23.039 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:18:23.251 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:18:23.383 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:18:23.375+0000 7fd40ffe0a40 -1 osd.3 0 log_to_monitors true 2026-02-24T18:18:24.252 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:18:24.252 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:18:24.253 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:18:24.254 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:18:24.254 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:18:24.254 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:18:24.457 INFO:tasks.workunit.client.0.trial172.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1294854746,v1:127.0.0.1:6827/1294854746] [v2:127.0.0.1:6828/1294854746,v1:127.0.0.1:6829/1294854746] exists,up 79d5a161-1bb7-472e-8989-c86f625396ef 2026-02-24T18:18:24.458 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:18:24.458 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:18:24.458 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:18:24.458 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:48: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-24T18:18:24.459 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:48: TEST_bluestore: osd_pid3=96415 2026-02-24T18:18:24.459 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:50: TEST_bluestore: sleep 5 2026-02-24T18:18:29.460 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:52: TEST_bluestore: create_pool foo 16 2026-02-24T18:18:29.461 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 2026-02-24T18:18:29.728 INFO:tasks.workunit.client.0.trial172.stderr:pool 'foo' already exists 2026-02-24T18:18:29.738 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T18:18:30.739 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:55: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-24T18:18:30.760 INFO:tasks.workunit.client.0.trial172.stdout:hints = 1 2026-02-24T18:18:30.760 INFO:tasks.workunit.client.0.trial172.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-24T18:18:30.761 INFO:tasks.workunit.client.0.trial172.stdout:Object prefix: benchmark_data_trial172_97465 2026-02-24T18:18:30.761 INFO:tasks.workunit.client.0.trial172.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T18:18:30.761 INFO:tasks.workunit.client.0.trial172.stdout: 0 0 0 0 0 0 - 0 2026-02-24T18:18:31.760 INFO:tasks.workunit.client.0.trial172.stdout: 1 16 10921 10905 42.5943 42.5977 0.00125843 0.00146476 2026-02-24T18:18:32.760 INFO:tasks.workunit.client.0.trial172.stdout: 2 16 21846 21830 42.6333 42.6758 0.00147185 0.00146398 2026-02-24T18:18:33.760 INFO:tasks.workunit.client.0.trial172.stdout: 3 16 32768 32752 42.6424 42.6641 0.001227 0.00146383 2026-02-24T18:18:34.760 INFO:tasks.workunit.client.0.trial172.stdout: 4 16 43681 43665 42.6383 42.6289 0.00103769 0.00146409 2026-02-24T18:18:35.760 INFO:tasks.workunit.client.0.trial172.stdout: 5 16 51951 51935 40.571 32.3047 0.0625621 0.00151477 2026-02-24T18:18:36.760 INFO:tasks.workunit.client.0.trial172.stdout: 6 16 62950 62934 40.9695 42.9648 0.00160174 0.0015239 2026-02-24T18:18:37.761 INFO:tasks.workunit.client.0.trial172.stdout: 7 16 73603 73587 41.061 41.6133 0.000975868 0.00152045 2026-02-24T18:18:38.760 INFO:tasks.workunit.client.0.trial172.stdout: 8 16 84388 84372 41.1941 42.1289 0.00115625 0.00151566 2026-02-24T18:18:39.761 INFO:tasks.workunit.client.0.trial172.stdout: 9 16 95141 95125 41.2837 42.0039 0.00158538 0.00151233 2026-02-24T18:18:40.761 INFO:tasks.workunit.client.0.trial172.stdout: 10 16 105970 105954 41.3851 42.3008 0.00124869 0.00150863 2026-02-24T18:18:41.761 INFO:tasks.workunit.client.0.trial172.stdout: 11 16 107477 107461 38.1579 5.88672 0.0857595 0.00162924 2026-02-24T18:18:42.761 INFO:tasks.workunit.client.0.trial172.stdout: 12 16 113277 113261 36.8659 22.6562 0.00118494 0.00169379 2026-02-24T18:18:43.761 INFO:tasks.workunit.client.0.trial172.stdout: 13 16 124147 124131 37.2961 42.4609 0.00149874 0.00167422 2026-02-24T18:18:44.761 INFO:tasks.workunit.client.0.trial172.stdout: 14 16 135091 135075 37.6854 42.75 0.00149148 0.00165693 2026-02-24T18:18:45.761 INFO:tasks.workunit.client.0.trial172.stdout: 15 16 144729 144713 37.6827 37.6484 0.00154648 0.00165701 2026-02-24T18:18:46.761 INFO:tasks.workunit.client.0.trial172.stdout: 16 15 154205 154190 37.6411 37.0195 0.0013402 0.00165888 2026-02-24T18:18:47.761 INFO:tasks.workunit.client.0.trial172.stdout: 17 16 159250 159234 36.5858 19.7031 0.136782 0.00168939 2026-02-24T18:18:48.761 INFO:tasks.workunit.client.0.trial172.stdout: 18 15 164038 164023 35.5925 18.707 0.00194196 0.00175444 2026-02-24T18:18:49.762 INFO:tasks.workunit.client.0.trial172.stdout: 19 16 173240 173224 35.6107 35.9414 0.00176621 0.00175356 2026-02-24T18:18:50.762 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:18:50.759619+0000 min lat: 0.000417102 max lat: 0.301278 avg lat: 0.00175331 2026-02-24T18:18:50.762 INFO:tasks.workunit.client.0.trial172.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T18:18:50.762 INFO:tasks.workunit.client.0.trial172.stdout: 20 16 182383 182367 35.6158 35.7148 0.00158078 0.00175331 2026-02-24T18:18:51.762 INFO:tasks.workunit.client.0.trial172.stdout: 21 16 191483 191467 35.6124 35.5469 0.00172065 0.00175348 2026-02-24T18:18:52.762 INFO:tasks.workunit.client.0.trial172.stdout: 22 16 200633 200617 35.6181 35.7422 0.000836851 0.00175315 2026-02-24T18:18:53.762 INFO:tasks.workunit.client.0.trial172.stdout: 23 16 206837 206821 35.1231 24.2344 0.235729 0.00176577 2026-02-24T18:18:54.762 INFO:tasks.workunit.client.0.trial172.stdout: 24 16 206986 206970 33.6839 0.582031 0.1127 0.00185084 2026-02-24T18:18:55.762 INFO:tasks.workunit.client.0.trial172.stdout: 25 16 210725 210709 32.9207 14.6055 0.00149243 0.00189696 2026-02-24T18:18:56.762 INFO:tasks.workunit.client.0.trial172.stdout: 26 16 219937 219921 33.0385 35.9844 0.00212659 0.00189021 2026-02-24T18:18:57.762 INFO:tasks.workunit.client.0.trial172.stdout: 27 16 229062 229046 33.1349 35.6445 0.00203556 0.00188471 2026-02-24T18:18:58.762 INFO:tasks.workunit.client.0.trial172.stdout: 28 15 238242 238227 33.2322 35.8633 0.00151884 0.00187919 2026-02-24T18:18:59.762 INFO:tasks.workunit.client.0.trial172.stdout: 29 16 247300 247284 33.3061 35.3789 0.00164309 0.00187501 2026-02-24T18:19:00.762 INFO:tasks.workunit.client.0.trial172.stdout:Total time run: 30.0015 2026-02-24T18:19:00.763 INFO:tasks.workunit.client.0.trial172.stdout:Total writes made: 256219 2026-02-24T18:19:00.763 INFO:tasks.workunit.client.0.trial172.stdout:Write size: 4096 2026-02-24T18:19:00.763 INFO:tasks.workunit.client.0.trial172.stdout:Object size: 4096 2026-02-24T18:19:00.763 INFO:tasks.workunit.client.0.trial172.stdout:Bandwidth (MB/sec): 33.3602 2026-02-24T18:19:00.763 INFO:tasks.workunit.client.0.trial172.stdout:Stddev Bandwidth: 11.5965 2026-02-24T18:19:00.763 INFO:tasks.workunit.client.0.trial172.stdout:Max bandwidth (MB/sec): 42.9648 2026-02-24T18:19:00.763 INFO:tasks.workunit.client.0.trial172.stdout:Min bandwidth (MB/sec): 0.582031 2026-02-24T18:19:00.763 INFO:tasks.workunit.client.0.trial172.stdout:Average IOPS: 8540 2026-02-24T18:19:00.763 INFO:tasks.workunit.client.0.trial172.stdout:Stddev IOPS: 2968.71 2026-02-24T18:19:00.764 INFO:tasks.workunit.client.0.trial172.stdout:Max IOPS: 10999 2026-02-24T18:19:00.764 INFO:tasks.workunit.client.0.trial172.stdout:Min IOPS: 149 2026-02-24T18:19:00.764 INFO:tasks.workunit.client.0.trial172.stdout:Average Latency(s): 0.001872 2026-02-24T18:19:00.764 INFO:tasks.workunit.client.0.trial172.stdout:Stddev Latency(s): 0.00609591 2026-02-24T18:19:00.764 INFO:tasks.workunit.client.0.trial172.stdout:Max latency(s): 0.358945 2026-02-24T18:19:00.764 INFO:tasks.workunit.client.0.trial172.stdout:Min latency(s): 0.000417102 2026-02-24T18:19:00.766 INFO:tasks.workunit.client.0.trial172.stdout:after bench 2026-02-24T18:19:00.766 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:57: TEST_bluestore: echo 'after bench' 2026-02-24T18:19:00.767 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: kill 89666 2026-02-24T18:19:00.767 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: sleep 1 2026-02-24T18:19:01.768 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: kill 89666 2026-02-24T18:19:01.768 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 60: kill: (89666) - No such process 2026-02-24T18:19:01.768 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:61: TEST_bluestore: ceph osd down 0 2026-02-24T18:19:01.985 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already down. 2026-02-24T18:19:01.997 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: kill 92121 2026-02-24T18:19:01.997 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: sleep 1 2026-02-24T18:19:02.999 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: kill 92121 2026-02-24T18:19:02.999 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 62: kill: (92121) - No such process 2026-02-24T18:19:02.999 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:63: TEST_bluestore: ceph osd down 1 2026-02-24T18:19:03.215 INFO:tasks.workunit.client.0.trial172.stderr:osd.1 is already down. 2026-02-24T18:19:03.225 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: kill 94268 2026-02-24T18:19:03.225 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: sleep 1 2026-02-24T18:19:04.226 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: kill 94268 2026-02-24T18:19:04.226 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 64: kill: (94268) - No such process 2026-02-24T18:19:04.227 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:65: TEST_bluestore: ceph osd down 2 2026-02-24T18:19:04.442 INFO:tasks.workunit.client.0.trial172.stderr:osd.2 is already down. 2026-02-24T18:19:04.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:66: TEST_bluestore: kill 96415 2026-02-24T18:19:04.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:66: TEST_bluestore: sleep 1 2026-02-24T18:19:05.453 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:66: TEST_bluestore: kill 96415 2026-02-24T18:19:05.454 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 66: kill: (96415) - No such process 2026-02-24T18:19:05.454 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:67: TEST_bluestore: ceph osd down 3 2026-02-24T18:19:05.718 INFO:tasks.workunit.client.0.trial172.stderr:osd.3 is already down. 2026-02-24T18:19:05.729 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:70: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-24T18:19:11.003 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:19:11.042 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:71: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-24T18:19:15.223 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:19:15.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:72: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-24T18:19:19.519 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:19:19.558 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:73: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-24T18:19:23.763 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:19:23.802 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:75: TEST_bluestore: truncate td/osd-bluefs-volume-ops/0/block -s 4294967296 2026-02-24T18:19:23.804 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:76: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 bluefs-bdev-expand 2026-02-24T18:19:23.820 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:19:27.428 INFO:tasks.workunit.client.0.trial172.stdout:0 : device size 0x20000000(512 MiB) : using 0x16d00000(365 MiB) 2026-02-24T18:19:27.429 INFO:tasks.workunit.client.0.trial172.stdout:1 : device size 0x40000000(1 GiB) : using 0xe00000(14 MiB) 2026-02-24T18:19:27.429 INFO:tasks.workunit.client.0.trial172.stdout:2 : device size 0x100000000(4 GiB) : using 0xaedf0000(2.7 GiB) : bluefs used 0x0(0 B) : non-bluefs used 0xaedf0000(2.7 GiB) 2026-02-24T18:19:27.429 INFO:tasks.workunit.client.0.trial172.stdout:Expanding DB/WAL... 2026-02-24T18:19:27.429 INFO:tasks.workunit.client.0.trial172.stdout:0 : nothing to do, skipped 2026-02-24T18:19:27.429 INFO:tasks.workunit.client.0.trial172.stdout:1 : nothing to do, skipped 2026-02-24T18:19:27.429 INFO:tasks.workunit.client.0.trial172.stdout:2 : Expanding to 0x100000000(4 GiB) 2026-02-24T18:19:27.430 INFO:tasks.workunit.client.0.trial172.stdout:2 : size updated to 0x100000000(4 GiB) 2026-02-24T18:19:31.325 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:77: TEST_bluestore: truncate td/osd-bluefs-volume-ops/1/block -s 11811160064 2026-02-24T18:19:31.326 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:78: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 bluefs-bdev-expand 2026-02-24T18:19:31.345 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:19:34.905 INFO:tasks.workunit.client.0.trial172.stdout:0 : device size 0x20000000(512 MiB) : using 0x16d00000(365 MiB) 2026-02-24T18:19:34.905 INFO:tasks.workunit.client.0.trial172.stdout:1 : device size 0x40000000(1 GiB) : using 0xe00000(14 MiB) 2026-02-24T18:19:34.905 INFO:tasks.workunit.client.0.trial172.stdout:2 : device size 0x2c0000000(11 GiB) : using 0x26efdb000(9.7 GiB) : bluefs used 0x0(0 B) : non-bluefs used 0x26efdb000(9.7 GiB) 2026-02-24T18:19:34.905 INFO:tasks.workunit.client.0.trial172.stdout:Expanding DB/WAL... 2026-02-24T18:19:34.906 INFO:tasks.workunit.client.0.trial172.stdout:0 : nothing to do, skipped 2026-02-24T18:19:34.906 INFO:tasks.workunit.client.0.trial172.stdout:1 : nothing to do, skipped 2026-02-24T18:19:34.906 INFO:tasks.workunit.client.0.trial172.stdout:2 : Expanding to 0x2c0000000(11 GiB) 2026-02-24T18:19:34.906 INFO:tasks.workunit.client.0.trial172.stdout:2 : size updated to 0x2c0000000(11 GiB) 2026-02-24T18:19:38.760 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:79: TEST_bluestore: truncate td/osd-bluefs-volume-ops/2/block -s 4295099392 2026-02-24T18:19:38.761 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:80: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 bluefs-bdev-expand 2026-02-24T18:19:38.780 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:19:42.390 INFO:tasks.workunit.client.0.trial172.stdout:0 : device size 0x20000000(512 MiB) : using 0x16d00000(365 MiB) 2026-02-24T18:19:42.391 INFO:tasks.workunit.client.0.trial172.stdout:1 : device size 0x40000000(1 GiB) : using 0xe00000(14 MiB) 2026-02-24T18:19:42.391 INFO:tasks.workunit.client.0.trial172.stdout:2 : device size 0x100020000(4.0 GiB) : using 0xaeec3000(2.7 GiB) : bluefs used 0x0(0 B) : non-bluefs used 0xaeec3000(2.7 GiB) 2026-02-24T18:19:42.391 INFO:tasks.workunit.client.0.trial172.stdout:Expanding DB/WAL... 2026-02-24T18:19:42.391 INFO:tasks.workunit.client.0.trial172.stdout:0 : nothing to do, skipped 2026-02-24T18:19:42.391 INFO:tasks.workunit.client.0.trial172.stdout:1 : nothing to do, skipped 2026-02-24T18:19:42.391 INFO:tasks.workunit.client.0.trial172.stdout:2 : Expanding to 0x100020000(4.0 GiB) 2026-02-24T18:19:42.392 INFO:tasks.workunit.client.0.trial172.stdout:2 : size updated to 0x100020000(4.0 GiB) 2026-02-24T18:19:46.365 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:81: TEST_bluestore: truncate td/osd-bluefs-volume-ops/3/block -s 4293918720 2026-02-24T18:19:46.366 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:82: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 bluefs-bdev-expand 2026-02-24T18:19:46.385 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:19:49.945 INFO:tasks.workunit.client.0.trial172.stdout:0 : device size 0x20000000(512 MiB) : using 0x16d00000(365 MiB) 2026-02-24T18:19:49.945 INFO:tasks.workunit.client.0.trial172.stdout:1 : device size 0x40000000(1 GiB) : using 0xe00000(14 MiB) 2026-02-24T18:19:49.946 INFO:tasks.workunit.client.0.trial172.stdout:2 : device size 0xfff00000(4.0 GiB) : using 0xaee04000(2.7 GiB) : bluefs used 0x0(0 B) : non-bluefs used 0xaee04000(2.7 GiB) 2026-02-24T18:19:49.946 INFO:tasks.workunit.client.0.trial172.stdout:Expanding DB/WAL... 2026-02-24T18:19:49.946 INFO:tasks.workunit.client.0.trial172.stdout:0 : nothing to do, skipped 2026-02-24T18:19:49.946 INFO:tasks.workunit.client.0.trial172.stdout:1 : nothing to do, skipped 2026-02-24T18:19:49.946 INFO:tasks.workunit.client.0.trial172.stdout:2 : Expanding to 0xfff00000(4.0 GiB) 2026-02-24T18:19:49.946 INFO:tasks.workunit.client.0.trial172.stdout:2 : size updated to 0xfff00000(4.0 GiB) 2026-02-24T18:19:53.881 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:85: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-24T18:19:54.827 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:19:54.864 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:86: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-24T18:19:55.799 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:19:55.837 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:87: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-24T18:19:56.763 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:19:56.800 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:88: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-24T18:19:57.775 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:19:57.812 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:90: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 bluefs-bdev-sizes 2026-02-24T18:19:57.831 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:19:58.027 INFO:tasks.workunit.client.0.trial172.stdout:0 : device size 0x20000000(512 MiB) : using 0x500000(5 MiB) 2026-02-24T18:19:58.027 INFO:tasks.workunit.client.0.trial172.stdout:1 : device size 0x40000000(1 GiB) : using 0x4100000(65 MiB) 2026-02-24T18:19:58.027 INFO:tasks.workunit.client.0.trial172.stdout:2 : device size 0x100000000(4 GiB) : using 0x2edf0000(750 MiB) : bluefs used 0x0(0 B) : non-bluefs used 0x2edf0000(750 MiB) 2026-02-24T18:19:58.214 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:92: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.wal --dev-target td/osd-bluefs-volume-ops/0/block.db --command bluefs-bdev-migrate 2026-02-24T18:19:58.226 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:19:58.755 INFO:tasks.workunit.client.0.trial172.stdout: device removed:0 td/osd-bluefs-volume-ops/0/block.wal 2026-02-24T18:19:58.758 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:97: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-24T18:19:59.543 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:19:59.580 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:100: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block.db --dev-target td/osd-bluefs-volume-ops/1/block --command bluefs-bdev-migrate 2026-02-24T18:19:59.597 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:20:00.203 INFO:tasks.workunit.client.0.trial172.stdout: device removed:1 td/osd-bluefs-volume-ops/1/block.db 2026-02-24T18:20:00.208 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:105: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-24T18:20:01.039 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:20:01.072 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:108: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block.wal --devs-source td/osd-bluefs-volume-ops/2/block.db --dev-target td/osd-bluefs-volume-ops/2/block --command bluefs-bdev-migrate 2026-02-24T18:20:01.090 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:20:01.711 INFO:tasks.workunit.client.0.trial172.stdout: device removed:1 td/osd-bluefs-volume-ops/2/block.db 2026-02-24T18:20:01.711 INFO:tasks.workunit.client.0.trial172.stdout: device removed:0 td/osd-bluefs-volume-ops/2/block.wal 2026-02-24T18:20:01.716 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:114: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-24T18:20:02.435 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:20:02.464 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:117: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block.db --dev-target td/osd-bluefs-volume-ops/3/block.wal --command bluefs-bdev-migrate 2026-02-24T18:20:02.481 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:20:02.611 INFO:tasks.workunit.client.0.trial172.stderr:Migrate to WAL device isn't supported. 2026-02-24T18:20:02.613 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:123: TEST_bluestore: '[' 1 -eq 0 ']' 2026-02-24T18:20:02.613 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:126: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-24T18:20:03.495 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:20:03.532 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:129: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block.wal --dev-target td/osd-bluefs-volume-ops/3/block --command bluefs-bdev-migrate 2026-02-24T18:20:03.550 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:20:04.119 INFO:tasks.workunit.client.0.trial172.stdout: device removed:0 td/osd-bluefs-volume-ops/3/block.wal 2026-02-24T18:20:04.123 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:134: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-24T18:20:04.943 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:20:04.973 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:136: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block --dev-target td/osd-bluefs-volume-ops/3/block.db --command bluefs-bdev-migrate 2026-02-24T18:20:04.991 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:20:05.358 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:141: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-24T18:20:06.163 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:20:06.195 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:143: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 0 2026-02-24T18:20:06.195 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:20:06.195 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:20:06.195 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-24T18:20:06.195 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:20:06.195 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-24T18:20:06.195 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:20:06.195 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:20:06.196 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:20:06.196 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:20:06.196 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-24T18:20:06.196 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-24T18:20:06.196 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:20:06.196 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:20:06.196 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:20:06.196 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:20:06.196 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:20:06.196 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:20:06.196 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:20:06.196 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:20:06.197 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:20:06.197 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:20:06.197 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:20:06.197 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:20:06.197 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:20:06.197 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:20:06.197 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:20:06.197 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:20:06.197 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:20:06.197 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:20:06.197 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:20:06.197 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:20:06.198 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-24T18:20:06.198 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:20:06.199 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-24T18:20:06.199 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:20:06.199 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-24T18:20:06.199 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-24T18:20:06.199 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:20:06.200 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:20:06.200 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:20:06.216 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:06.210+0000 7f5626defa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:20:06.217 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:06.210+0000 7f5626defa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:20:06.218 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:06.210+0000 7f5626defa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:20:06.411 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-24T18:20:06.411 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:20:06.412 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:20:06.412 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:20:06.412 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:20:06.412 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:20:06.412 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:20:06.413 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:20:06.413 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:20:06.413 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:20:06.580 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:06.574+0000 7f5626defa40 -1 Falling back to public interface 2026-02-24T18:20:06.627 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:20:07.628 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:20:07.628 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:20:07.629 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:20:07.630 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:20:07.630 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:20:07.630 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:20:07.841 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:20:08.397 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:08.390+0000 7f5626defa40 -1 osd.0 23 log_to_monitors true 2026-02-24T18:20:08.842 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:20:08.842 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:20:08.843 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:20:08.843 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:20:08.844 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:20:08.844 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:20:09.060 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 31 up_thru 31 down_at 23 last_clean_interval [5,23) [v2:127.0.0.1:6802/596753141,v1:127.0.0.1:6803/596753141] [v2:127.0.0.1:6804/596753141,v1:127.0.0.1:6805/596753141] exists,up 410d8f94-9514-413c-9295-e8e271a743d8 2026-02-24T18:20:09.061 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:20:09.062 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:20:09.062 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:20:09.062 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:144: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-24T18:20:09.065 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:144: TEST_bluestore: osd_pid0=108115 2026-02-24T18:20:09.065 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:145: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 1 2026-02-24T18:20:09.066 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:20:09.066 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:20:09.066 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T18:20:09.066 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:20:09.066 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-24T18:20:09.066 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:20:09.066 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:20:09.066 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:20:09.066 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:20:09.067 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-24T18:20:09.067 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-24T18:20:09.067 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:20:09.067 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:20:09.067 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:20:09.067 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:20:09.067 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:20:09.067 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:20:09.067 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:20:09.067 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:20:09.067 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:20:09.068 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:20:09.068 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:20:09.068 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:20:09.068 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:20:09.068 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:20:09.068 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:20:09.068 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:20:09.068 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:20:09.068 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:20:09.068 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:20:09.069 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:20:09.069 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-24T18:20:09.069 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:20:09.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T18:20:09.070 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:20:09.070 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/1/whoami 2026-02-24T18:20:09.071 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T18:20:09.071 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:20:09.071 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:20:09.071 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:20:09.084 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:09.078+0000 7fe06c92fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:20:09.085 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:09.078+0000 7fe06c92fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:20:09.086 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:09.078+0000 7fe06c92fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:20:09.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T18:20:09.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:20:09.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:20:09.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:20:09.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:20:09.288 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:20:09.288 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:20:09.288 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:20:09.289 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:20:09.289 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:20:09.497 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:20:09.589 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:09.582+0000 7fe06c92fa40 -1 Falling back to public interface 2026-02-24T18:20:10.498 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:20:10.498 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:20:10.498 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:20:10.498 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:20:10.498 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:20:10.498 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:20:10.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:20:11.326 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:11.318+0000 7fe06c92fa40 -1 osd.1 24 log_to_monitors true 2026-02-24T18:20:11.719 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:20:11.720 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:20:11.720 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:20:11.721 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:20:11.721 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:20:11.721 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:20:11.934 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 35 up_thru 35 down_at 25 last_clean_interval [9,24) [v2:127.0.0.1:6810/2135964921,v1:127.0.0.1:6811/2135964921] [v2:127.0.0.1:6812/2135964921,v1:127.0.0.1:6813/2135964921] exists,up f4b079dc-13d9-4358-9502-d9202bc6b891 2026-02-24T18:20:11.935 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:20:11.935 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:20:11.935 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:20:11.935 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:146: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-24T18:20:11.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:146: TEST_bluestore: osd_pid1=109056 2026-02-24T18:20:11.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:147: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 2 2026-02-24T18:20:11.938 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:20:11.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:20:11.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T18:20:11.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:20:11.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-24T18:20:11.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:20:11.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:20:11.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:20:11.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:20:11.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-24T18:20:11.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-24T18:20:11.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:20:11.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:20:11.940 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:20:11.940 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:20:11.940 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:20:11.940 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:20:11.940 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:20:11.940 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:20:11.940 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:20:11.940 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:20:11.941 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:20:11.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:20:11.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:20:11.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:20:11.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:20:11.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:20:11.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:20:11.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:20:11.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:20:11.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:20:11.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-24T18:20:11.943 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:20:11.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T18:20:11.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:20:11.944 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/2/whoami 2026-02-24T18:20:11.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T18:20:11.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:20:11.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:20:11.944 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:20:11.959 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:11.954+0000 7f4563fc2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:20:11.960 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:11.954+0000 7f4563fc2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:20:11.961 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:11.954+0000 7f4563fc2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:20:12.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T18:20:12.154 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:20:12.154 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:20:12.154 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:20:12.154 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:20:12.154 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:20:12.154 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:20:12.155 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:20:12.155 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:20:12.155 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:20:12.372 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:20:12.388 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:12.382+0000 7f4563fc2a40 -1 Falling back to public interface 2026-02-24T18:20:13.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:20:13.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:20:13.374 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:20:13.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:20:13.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:20:13.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:20:13.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:20:14.082 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:14.074+0000 7f4563fc2a40 -1 osd.2 26 log_to_monitors true 2026-02-24T18:20:14.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:20:14.596 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:20:14.596 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:20:14.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:20:14.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:20:14.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:20:14.821 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 39 up_thru 39 down_at 27 last_clean_interval [13,26) [v2:127.0.0.1:6818/2562337203,v1:127.0.0.1:6819/2562337203] [v2:127.0.0.1:6820/2562337203,v1:127.0.0.1:6821/2562337203] exists,up c77df6a7-2ace-45f0-9cdb-0238c4cea4fe 2026-02-24T18:20:14.821 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:20:14.821 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:20:14.821 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:20:14.822 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:148: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:148: TEST_bluestore: osd_pid2=109997 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:149: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 3 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:20:14.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:20:14.824 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:20:14.824 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:20:14.824 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:20:14.824 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:20:14.824 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:20:14.824 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:20:14.825 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:20:14.825 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:20:14.825 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:20:14.825 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:20:14.825 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:20:14.825 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:20:14.825 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:20:14.825 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:20:14.825 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:20:14.825 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:20:14.825 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:20:14.825 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-24T18:20:14.826 INFO:tasks.workunit.client.0.trial172.stdout:start osd.3 2026-02-24T18:20:14.826 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-24T18:20:14.827 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:20:14.827 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/3/whoami 2026-02-24T18:20:14.828 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-24T18:20:14.828 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:20:14.829 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:20:14.829 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:20:14.842 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:14.834+0000 7f558b45fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:20:14.843 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:14.838+0000 7f558b45fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:20:14.844 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:14.838+0000 7f558b45fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:20:15.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-24T18:20:15.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:20:15.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T18:20:15.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:20:15.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:20:15.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:20:15.045 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:20:15.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:20:15.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:20:15.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:20:15.257 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:20:15.264 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:15.258+0000 7f558b45fa40 -1 Falling back to public interface 2026-02-24T18:20:16.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:20:16.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:20:16.260 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:20:16.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:20:16.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:20:16.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:20:16.481 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:20:17.044 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:20:17.038+0000 7f558b45fa40 -1 osd.3 28 log_to_monitors true 2026-02-24T18:20:17.482 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:20:17.483 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:20:17.483 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:20:17.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:20:17.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:20:17.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:20:17.693 INFO:tasks.workunit.client.0.trial172.stdout:osd.3 up in weight 1 up_from 42 up_thru 42 down_at 29 last_clean_interval [18,28) [v2:127.0.0.1:6826/3832148744,v1:127.0.0.1:6827/3832148744] [v2:127.0.0.1:6828/3832148744,v1:127.0.0.1:6829/3832148744] exists,up 79d5a161-1bb7-472e-8989-c86f625396ef 2026-02-24T18:20:17.694 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:20:17.694 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:20:17.694 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:20:17.694 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:150: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-24T18:20:17.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:150: TEST_bluestore: osd_pid3=110933 2026-02-24T18:20:17.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:152: TEST_bluestore: wait_for_clean 2026-02-24T18:20:17.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:20:17.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:20:17.696 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:20:17.697 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T18:20:17.697 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:20:17.697 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:20:17.698 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:20:17.698 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:20:17.698 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:20:17.791 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T18:20:17.791 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:20:17.792 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:20:17.792 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:20:17.792 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:20:17.792 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:20:18.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:20:18.018 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:20:18.018 INFO:tasks.workunit.client.0.trial172.stderr:2 2026-02-24T18:20:18.018 INFO:tasks.workunit.client.0.trial172.stderr:3' 2026-02-24T18:20:18.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:20:18.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:20:18.019 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:20:18.097 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=133143986180 2026-02-24T18:20:18.097 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 133143986180 2026-02-24T18:20:18.097 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-133143986180' 2026-02-24T18:20:18.097 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:20:18.097 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:20:18.177 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=150323855364 2026-02-24T18:20:18.177 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 150323855364 2026-02-24T18:20:18.177 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-133143986180 1-150323855364' 2026-02-24T18:20:18.177 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:20:18.177 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:20:18.255 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=167503724547 2026-02-24T18:20:18.255 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 167503724547 2026-02-24T18:20:18.255 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-133143986180 1-150323855364 2-167503724547' 2026-02-24T18:20:18.255 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:20:18.255 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T18:20:18.337 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=180388626435 2026-02-24T18:20:18.337 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 180388626435 2026-02-24T18:20:18.337 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-133143986180 1-150323855364 2-167503724547 3-180388626435' 2026-02-24T18:20:18.337 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:20:18.338 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-133143986180 2026-02-24T18:20:18.338 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:20:18.338 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:20:18.339 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-133143986180 2026-02-24T18:20:18.340 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:20:18.341 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=133143986180 2026-02-24T18:20:18.341 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 133143986180' 2026-02-24T18:20:18.341 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 133143986180 2026-02-24T18:20:18.342 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:20:18.567 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 133143986178 -lt 133143986180 2026-02-24T18:20:18.567 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:20:19.568 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:20:19.568 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:20:19.782 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 133143986181 -lt 133143986180 2026-02-24T18:20:19.782 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:20:19.783 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-150323855364 2026-02-24T18:20:19.783 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:20:19.784 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:20:19.785 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-150323855364 2026-02-24T18:20:19.785 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:20:19.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=150323855364 2026-02-24T18:20:19.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 150323855364' 2026-02-24T18:20:19.787 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 150323855364 2026-02-24T18:20:19.787 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:20:19.994 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 150323855364 -lt 150323855364 2026-02-24T18:20:19.994 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:20:19.995 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-167503724547 2026-02-24T18:20:19.995 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:20:19.996 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:20:19.997 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-167503724547 2026-02-24T18:20:19.997 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:20:19.998 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=167503724547 2026-02-24T18:20:19.998 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 167503724547' 2026-02-24T18:20:19.999 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 167503724547 2026-02-24T18:20:19.999 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:20:20.226 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-24T18:20:20.226 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:20:20.227 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-180388626435 2026-02-24T18:20:20.227 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:20:20.228 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T18:20:20.229 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-180388626435 2026-02-24T18:20:20.229 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:20:20.230 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=180388626435 2026-02-24T18:20:20.230 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 180388626435' 2026-02-24T18:20:20.231 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.3 seq 180388626435 2026-02-24T18:20:20.231 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T18:20:20.461 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 180388626435 -lt 180388626435 2026-02-24T18:20:20.462 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:20:20.462 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:20:20.463 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:20:20.750 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-24T18:20:20.750 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:20:20.751 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:20:20.752 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:20:20.752 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:20:20.752 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:20:20.752 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:20:20.752 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:20:20.969 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-24T18:20:20.970 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:20:20.970 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:20:20.970 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:20:21.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-24T18:20:21.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T18:20:21.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T18:20:21.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:155: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-24T18:20:21.279 INFO:tasks.workunit.client.0.trial172.stdout:hints = 1 2026-02-24T18:20:21.279 INFO:tasks.workunit.client.0.trial172.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-24T18:20:21.279 INFO:tasks.workunit.client.0.trial172.stdout:Object prefix: benchmark_data_trial172_112250 2026-02-24T18:20:21.280 INFO:tasks.workunit.client.0.trial172.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T18:20:21.280 INFO:tasks.workunit.client.0.trial172.stdout: 0 0 0 0 0 0 - 0 2026-02-24T18:20:22.279 INFO:tasks.workunit.client.0.trial172.stdout: 1 16 10136 10120 39.5281 39.5312 0.00141705 0.00157827 2026-02-24T18:20:23.279 INFO:tasks.workunit.client.0.trial172.stdout: 2 16 19827 19811 38.6903 37.8555 0.00174627 0.00161328 2026-02-24T18:20:24.279 INFO:tasks.workunit.client.0.trial172.stdout: 3 16 29471 29455 38.3498 37.6719 0.00151733 0.00162788 2026-02-24T18:20:25.279 INFO:tasks.workunit.client.0.trial172.stdout: 4 16 39080 39064 38.1454 37.5352 0.00137736 0.00163661 2026-02-24T18:20:26.279 INFO:tasks.workunit.client.0.trial172.stdout: 5 16 48741 48725 38.0633 37.7383 0.00197988 0.00164026 2026-02-24T18:20:27.280 INFO:tasks.workunit.client.0.trial172.stdout: 6 15 58338 58323 37.9676 37.4922 0.00134923 0.00164447 2026-02-24T18:20:28.280 INFO:tasks.workunit.client.0.trial172.stdout: 7 16 67824 67808 37.8362 37.0508 0.00179382 0.00165025 2026-02-24T18:20:29.280 INFO:tasks.workunit.client.0.trial172.stdout: 8 16 77362 77346 37.7636 37.2578 0.00135631 0.00165336 2026-02-24T18:20:30.280 INFO:tasks.workunit.client.0.trial172.stdout: 9 16 86873 86857 37.6953 37.1523 0.00134877 0.00165652 2026-02-24T18:20:31.280 INFO:tasks.workunit.client.0.trial172.stdout: 10 16 96386 96370 37.6415 37.1602 0.00132052 0.0016588 2026-02-24T18:20:32.280 INFO:tasks.workunit.client.0.trial172.stdout: 11 16 105702 105686 37.5275 36.3906 0.00177088 0.00166386 2026-02-24T18:20:33.280 INFO:tasks.workunit.client.0.trial172.stdout: 12 16 115133 115117 37.47 36.8398 0.00175717 0.00166642 2026-02-24T18:20:34.280 INFO:tasks.workunit.client.0.trial172.stdout: 13 16 124387 124371 37.3681 36.1484 0.00170328 0.00167099 2026-02-24T18:20:35.280 INFO:tasks.workunit.client.0.trial172.stdout: 14 16 133934 133918 37.3625 37.293 0.0015634 0.00167126 2026-02-24T18:20:36.280 INFO:tasks.workunit.client.0.trial172.stdout: 15 16 143476 143460 37.3564 37.2734 0.0017918 0.00167154 2026-02-24T18:20:37.280 INFO:tasks.workunit.client.0.trial172.stdout: 16 16 153031 153015 37.3542 37.3242 0.00194978 0.00167166 2026-02-24T18:20:38.280 INFO:tasks.workunit.client.0.trial172.stdout: 17 16 162445 162429 37.3199 36.7734 0.00112129 0.00167315 2026-02-24T18:20:39.280 INFO:tasks.workunit.client.0.trial172.stdout: 18 16 171965 171949 37.3124 37.1875 0.0016726 0.00167353 2026-02-24T18:20:40.280 INFO:tasks.workunit.client.0.trial172.stdout: 19 16 181614 181598 37.3322 37.6914 0.00167832 0.00167266 2026-02-24T18:20:41.281 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:20:41.279302+0000 min lat: 0.00056328 max lat: 0.0306857 avg lat: 0.00167794 2026-02-24T18:20:41.281 INFO:tasks.workunit.client.0.trial172.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T18:20:41.281 INFO:tasks.workunit.client.0.trial172.stdout: 20 16 190302 190286 37.1623 33.9375 0.0234617 0.00167794 2026-02-24T18:20:42.281 INFO:tasks.workunit.client.0.trial172.stdout: 21 16 190465 190449 35.423 0.636719 0.0408246 0.00176102 2026-02-24T18:20:43.281 INFO:tasks.workunit.client.0.trial172.stdout: 22 16 190792 190776 33.8709 1.27734 0.0466586 0.00184237 2026-02-24T18:20:44.281 INFO:tasks.workunit.client.0.trial172.stdout: 23 16 191013 190997 32.4358 0.863281 0.0520098 0.00192184 2026-02-24T18:20:45.281 INFO:tasks.workunit.client.0.trial172.stdout: 24 16 191185 191169 31.1123 0.671875 0.0914986 0.0020021 2026-02-24T18:20:46.281 INFO:tasks.workunit.client.0.trial172.stdout: 25 16 191231 191215 29.875 0.179688 0.0684724 0.00208773 2026-02-24T18:20:47.281 INFO:tasks.workunit.client.0.trial172.stdout: 26 16 191360 191344 28.7454 0.503906 0.150396 0.00216366 2026-02-24T18:20:48.281 INFO:tasks.workunit.client.0.trial172.stdout: 27 16 196267 196251 28.3906 19.168 0.00139017 0.00219993 2026-02-24T18:20:49.281 INFO:tasks.workunit.client.0.trial172.stdout: 28 16 206022 206006 28.7375 38.1055 0.00174009 0.00217337 2026-02-24T18:20:50.281 INFO:tasks.workunit.client.0.trial172.stdout: 29 16 215590 215574 29.0352 37.375 0.00126711 0.00215107 2026-02-24T18:20:51.281 INFO:tasks.workunit.client.0.trial172.stdout:Total time run: 30.0008 2026-02-24T18:20:51.281 INFO:tasks.workunit.client.0.trial172.stdout:Total writes made: 225142 2026-02-24T18:20:51.282 INFO:tasks.workunit.client.0.trial172.stdout:Write size: 4096 2026-02-24T18:20:51.282 INFO:tasks.workunit.client.0.trial172.stdout:Object size: 4096 2026-02-24T18:20:51.282 INFO:tasks.workunit.client.0.trial172.stdout:Bandwidth (MB/sec): 29.3146 2026-02-24T18:20:51.282 INFO:tasks.workunit.client.0.trial172.stdout:Stddev Bandwidth: 15.1333 2026-02-24T18:20:51.282 INFO:tasks.workunit.client.0.trial172.stdout:Max bandwidth (MB/sec): 39.5312 2026-02-24T18:20:51.282 INFO:tasks.workunit.client.0.trial172.stdout:Min bandwidth (MB/sec): 0.179688 2026-02-24T18:20:51.282 INFO:tasks.workunit.client.0.trial172.stdout:Average IOPS: 7504 2026-02-24T18:20:51.282 INFO:tasks.workunit.client.0.trial172.stdout:Stddev IOPS: 3874.13 2026-02-24T18:20:51.282 INFO:tasks.workunit.client.0.trial172.stdout:Max IOPS: 10120 2026-02-24T18:20:51.282 INFO:tasks.workunit.client.0.trial172.stdout:Min IOPS: 46 2026-02-24T18:20:51.282 INFO:tasks.workunit.client.0.trial172.stdout:Average Latency(s): 0.00213059 2026-02-24T18:20:51.283 INFO:tasks.workunit.client.0.trial172.stdout:Stddev Latency(s): 0.00955973 2026-02-24T18:20:51.283 INFO:tasks.workunit.client.0.trial172.stdout:Max latency(s): 0.835532 2026-02-24T18:20:51.283 INFO:tasks.workunit.client.0.trial172.stdout:Min latency(s): 0.00056328 2026-02-24T18:20:51.284 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:158: TEST_bluestore: kill 108115 2026-02-24T18:20:51.285 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:158: TEST_bluestore: sleep 1 2026-02-24T18:20:52.286 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:158: TEST_bluestore: kill 108115 2026-02-24T18:20:52.286 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 158: kill: (108115) - No such process 2026-02-24T18:20:52.287 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:159: TEST_bluestore: ceph osd down 0 2026-02-24T18:20:52.494 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already down. 2026-02-24T18:20:52.504 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:160: TEST_bluestore: kill 109056 2026-02-24T18:20:52.504 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:160: TEST_bluestore: sleep 1 2026-02-24T18:20:53.506 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:160: TEST_bluestore: kill 109056 2026-02-24T18:20:53.506 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 160: kill: (109056) - No such process 2026-02-24T18:20:53.506 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:161: TEST_bluestore: ceph osd down 1 2026-02-24T18:20:53.715 INFO:tasks.workunit.client.0.trial172.stderr:osd.1 is already down. 2026-02-24T18:20:53.725 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:162: TEST_bluestore: kill 109997 2026-02-24T18:20:53.725 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:162: TEST_bluestore: sleep 1 2026-02-24T18:20:54.726 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:162: TEST_bluestore: kill 109997 2026-02-24T18:20:54.726 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 162: kill: (109997) - No such process 2026-02-24T18:20:54.726 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:163: TEST_bluestore: ceph osd down 2 2026-02-24T18:20:54.936 INFO:tasks.workunit.client.0.trial172.stderr:osd.2 is already down. 2026-02-24T18:20:54.945 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:164: TEST_bluestore: kill 110933 2026-02-24T18:20:54.945 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:164: TEST_bluestore: sleep 1 2026-02-24T18:20:55.948 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:164: TEST_bluestore: kill 110933 2026-02-24T18:20:55.948 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 164: kill: (110933) - No such process 2026-02-24T18:20:55.948 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:165: TEST_bluestore: ceph osd down 3 2026-02-24T18:20:56.204 INFO:tasks.workunit.client.0.trial172.stderr:osd.3 is already down. 2026-02-24T18:20:56.213 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:168: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-24T18:20:59.767 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:20:59.808 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:170: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/0/wal count=512 bs=1M 2026-02-24T18:21:00.129 INFO:tasks.workunit.client.0.trial172.stderr:512+0 records in 2026-02-24T18:21:00.130 INFO:tasks.workunit.client.0.trial172.stderr:512+0 records out 2026-02-24T18:21:00.130 INFO:tasks.workunit.client.0.trial172.stderr:536870912 bytes (537 MB, 512 MiB) copied, 0.3206 s, 1.7 GB/s 2026-02-24T18:21:00.130 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:171: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --dev-target td/osd-bluefs-volume-ops/0/wal --command bluefs-bdev-new-wal 2026-02-24T18:21:00.147 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:21:03.447 INFO:tasks.workunit.client.0.trial172.stdout:WAL device added /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/0/wal 2026-02-24T18:21:03.471 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:175: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-24T18:21:07.131 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:21:07.182 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:178: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-24T18:21:11.043 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:21:11.092 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:180: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/1/db count=1024 bs=1M 2026-02-24T18:21:11.718 INFO:tasks.workunit.client.0.trial172.stderr:1024+0 records in 2026-02-24T18:21:11.718 INFO:tasks.workunit.client.0.trial172.stderr:1024+0 records out 2026-02-24T18:21:11.718 INFO:tasks.workunit.client.0.trial172.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.624954 s, 1.7 GB/s 2026-02-24T18:21:11.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:181: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --dev-target td/osd-bluefs-volume-ops/1/db --command bluefs-bdev-new-db 2026-02-24T18:21:11.735 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:21:15.799 INFO:tasks.workunit.client.0.trial172.stdout:DB device added /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/1/db 2026-02-24T18:21:15.822 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:185: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block --dev-target td/osd-bluefs-volume-ops/1/block.db --command bluefs-bdev-migrate 2026-02-24T18:21:15.835 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:21:19.970 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:190: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-24T18:21:23.850 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:21:23.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:193: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-24T18:21:27.210 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:21:27.247 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:195: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --command bluefs-bdev-new-db 2026-02-24T18:21:27.264 INFO:tasks.workunit.client.0.trial172.stdout:NOTICE: --dev-target option omitted, will allocate as a file 2026-02-24T18:21:27.264 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:21:29.886 INFO:tasks.workunit.client.0.trial172.stdout:DB device added 2026-02-24T18:21:29.906 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:198: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --command bluefs-bdev-new-wal 2026-02-24T18:21:29.918 INFO:tasks.workunit.client.0.trial172.stdout:NOTICE: --dev-target option omitted, will allocate as a file 2026-02-24T18:21:29.918 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:21:32.666 INFO:tasks.workunit.client.0.trial172.stdout:WAL device added 2026-02-24T18:21:32.686 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:201: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block --dev-target td/osd-bluefs-volume-ops/2/block.db --command bluefs-bdev-migrate 2026-02-24T18:21:32.702 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:21:36.057 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:206: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-24T18:21:39.674 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:21:39.723 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:209: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-24T18:21:43.278 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:21:43.316 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:211: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --command bluefs-bdev-new-wal 2026-02-24T18:21:43.333 INFO:tasks.workunit.client.0.trial172.stdout:NOTICE: --dev-target option omitted, will allocate as a file 2026-02-24T18:21:43.333 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:21:46.294 INFO:tasks.workunit.client.0.trial172.stdout:WAL device added 2026-02-24T18:21:46.317 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:214: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block.db --dev-target td/osd-bluefs-volume-ops/3/block --command bluefs-bdev-migrate 2026-02-24T18:21:46.334 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:21:49.834 INFO:tasks.workunit.client.0.trial172.stdout: device removed:1 td/osd-bluefs-volume-ops/3/block.db 2026-02-24T18:21:49.864 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:219: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-24T18:21:53.410 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:21:53.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:221: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 0 2026-02-24T18:21:53.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:21:53.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:21:53.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-24T18:21:53.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:21:53.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-24T18:21:53.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:21:53.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:21:53.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:21:53.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:21:53.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-24T18:21:53.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-24T18:21:53.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:21:53.451 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:21:53.451 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:21:53.451 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:21:53.451 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:21:53.451 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:21:53.451 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:21:53.451 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:21:53.451 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:21:53.452 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:21:53.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:21:53.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:21:53.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:21:53.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:21:53.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:21:53.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:21:53.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:21:53.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:21:53.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:21:53.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:21:53.452 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-24T18:21:53.452 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:21:53.453 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-24T18:21:53.453 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:21:53.453 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-24T18:21:53.453 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-24T18:21:53.454 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:21:53.454 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:21:53.454 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:21:53.475 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:21:53.470+0000 7ff80e7cba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:21:53.477 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:21:53.470+0000 7ff80e7cba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:21:53.478 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:21:53.470+0000 7ff80e7cba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:21:53.671 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-24T18:21:53.671 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:21:53.671 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:21:53.672 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:21:53.672 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:21:53.672 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:21:53.672 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:21:53.673 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:21:53.673 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:21:53.673 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:21:53.878 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:21:53.960 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:21:53.954+0000 7ff80e7cba40 -1 Falling back to public interface 2026-02-24T18:21:54.879 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:21:54.879 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:21:54.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:21:54.880 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:21:54.880 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:21:54.881 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:21:55.103 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:21:56.105 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:21:56.105 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:21:56.105 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:21:56.106 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:21:56.106 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:21:56.106 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:21:56.323 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:21:57.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:21:57.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:21:57.324 INFO:tasks.workunit.client.0.trial172.stdout:3 2026-02-24T18:21:57.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-24T18:21:57.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:21:57.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:21:57.548 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:21:58.549 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:21:58.549 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:21:58.549 INFO:tasks.workunit.client.0.trial172.stdout:4 2026-02-24T18:21:58.550 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-24T18:21:58.550 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:21:58.550 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:21:58.764 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:21:59.766 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:21:59.766 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:21:59.766 INFO:tasks.workunit.client.0.trial172.stdout:5 2026-02-24T18:21:59.767 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-24T18:21:59.767 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:21:59.767 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:21:59.983 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:00.984 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:00.985 INFO:tasks.workunit.client.0.trial172.stdout:6 2026-02-24T18:22:00.985 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:00.985 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-24T18:22:00.985 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:00.985 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:22:01.172 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:01.166+0000 7ff80e7cba40 -1 osd.0 44 log_to_monitors true 2026-02-24T18:22:01.206 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:02.207 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:02.207 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:02.207 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-24T18:22:02.207 INFO:tasks.workunit.client.0.trial172.stdout:7 2026-02-24T18:22:02.208 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:02.208 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:22:02.426 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 52 up_thru 52 down_at 44 last_clean_interval [31,44) [v2:127.0.0.1:6802/3295548651,v1:127.0.0.1:6803/3295548651] [v2:127.0.0.1:6804/3295548651,v1:127.0.0.1:6805/3295548651] exists,up 410d8f94-9514-413c-9295-e8e271a743d8 2026-02-24T18:22:02.427 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:22:02.427 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:22:02.427 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:22:02.428 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:222: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-24T18:22:02.431 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:222: TEST_bluestore: osd_pid0=118401 2026-02-24T18:22:02.431 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:223: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 1 2026-02-24T18:22:02.431 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:22:02.431 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:22:02.431 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T18:22:02.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:22:02.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-24T18:22:02.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:22:02.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:22:02.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:22:02.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:22:02.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-24T18:22:02.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-24T18:22:02.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:22:02.433 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:22:02.433 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:22:02.433 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:22:02.433 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:22:02.433 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:22:02.433 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:22:02.433 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:22:02.433 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:22:02.433 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:22:02.433 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:22:02.433 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:22:02.433 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:22:02.434 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:22:02.434 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:22:02.434 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:22:02.434 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:22:02.434 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:22:02.434 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:22:02.434 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:22:02.434 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-24T18:22:02.435 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:22:02.436 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T18:22:02.436 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:22:02.436 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/1/whoami 2026-02-24T18:22:02.436 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T18:22:02.436 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:22:02.436 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:22:02.436 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:22:02.455 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:02.450+0000 7f8ba7192a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:22:02.456 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:02.450+0000 7f8ba7192a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:22:02.457 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:02.450+0000 7f8ba7192a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:22:02.640 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T18:22:02.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:22:02.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:22:02.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:22:02.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:22:02.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:02.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:22:02.641 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:22:02.642 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:02.642 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:22:02.863 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:03.012 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:03.006+0000 7f8ba7192a40 -1 Falling back to public interface 2026-02-24T18:22:03.865 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:03.865 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:03.865 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:22:03.866 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:22:03.867 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:03.867 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:22:04.077 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:05.079 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:05.079 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:05.079 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:22:05.080 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:22:05.080 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:05.081 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:22:05.290 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:06.291 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:06.291 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:06.291 INFO:tasks.workunit.client.0.trial172.stdout:3 2026-02-24T18:22:06.292 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-24T18:22:06.292 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:06.292 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:22:06.500 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:07.502 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:07.502 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:07.502 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-24T18:22:07.503 INFO:tasks.workunit.client.0.trial172.stdout:4 2026-02-24T18:22:07.503 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:07.503 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:22:07.723 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:08.724 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:08.724 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:08.724 INFO:tasks.workunit.client.0.trial172.stdout:5 2026-02-24T18:22:08.725 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-24T18:22:08.725 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:08.725 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:22:08.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:09.941 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:09.941 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:09.941 INFO:tasks.workunit.client.0.trial172.stdout:6 2026-02-24T18:22:09.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-24T18:22:09.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:09.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:22:10.167 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:11.007 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:11.002+0000 7f8ba7192a40 -1 osd.1 45 log_to_monitors true 2026-02-24T18:22:11.168 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:11.168 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:11.168 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-24T18:22:11.168 INFO:tasks.workunit.client.0.trial172.stdout:7 2026-02-24T18:22:11.169 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:11.169 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:22:11.381 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 55 up_thru 55 down_at 46 last_clean_interval [35,45) [v2:127.0.0.1:6810/978712080,v1:127.0.0.1:6811/978712080] [v2:127.0.0.1:6812/978712080,v1:127.0.0.1:6813/978712080] exists,up f4b079dc-13d9-4358-9502-d9202bc6b891 2026-02-24T18:22:11.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:22:11.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:22:11.381 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:22:11.382 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:224: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-24T18:22:11.383 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:224: TEST_bluestore: osd_pid1=119472 2026-02-24T18:22:11.383 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:225: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 2 2026-02-24T18:22:11.383 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:22:11.383 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:22:11.383 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T18:22:11.383 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:22:11.383 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-24T18:22:11.384 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:22:11.384 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:22:11.384 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:22:11.384 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:22:11.384 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-24T18:22:11.384 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-24T18:22:11.384 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:22:11.384 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:22:11.384 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:22:11.384 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:22:11.384 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:22:11.384 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:22:11.384 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:22:11.384 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:22:11.384 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:22:11.385 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:22:11.385 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:22:11.385 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:22:11.385 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:22:11.385 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:22:11.385 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:22:11.385 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:22:11.385 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:22:11.385 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:22:11.385 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:22:11.386 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:22:11.386 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-24T18:22:11.386 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T18:22:11.386 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:22:11.387 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:22:11.387 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/2/whoami 2026-02-24T18:22:11.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T18:22:11.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:22:11.388 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:22:11.389 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:22:11.405 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:11.398+0000 7f70aa774a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:22:11.407 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:11.402+0000 7f70aa774a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:22:11.408 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:11.402+0000 7f70aa774a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:22:11.607 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T18:22:11.608 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:22:11.608 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:22:11.608 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:22:11.608 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:22:11.608 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:11.608 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:22:11.609 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:22:11.609 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:11.609 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:22:11.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:11.931 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:11.926+0000 7f70aa774a40 -1 Falling back to public interface 2026-02-24T18:22:12.822 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:12.822 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:12.823 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:22:12.823 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:22:12.824 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:12.824 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:22:13.042 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:14.043 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:14.044 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:14.044 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:22:14.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:22:14.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:14.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:22:14.265 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:15.266 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:15.267 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:15.267 INFO:tasks.workunit.client.0.trial172.stdout:3 2026-02-24T18:22:15.268 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-24T18:22:15.268 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:15.268 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:22:15.484 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:16.486 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:16.486 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:16.487 INFO:tasks.workunit.client.0.trial172.stdout:4 2026-02-24T18:22:16.487 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-24T18:22:16.487 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:16.488 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:22:16.705 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:17.706 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:17.706 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:17.706 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-24T18:22:17.706 INFO:tasks.workunit.client.0.trial172.stdout:5 2026-02-24T18:22:17.706 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:17.706 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:22:17.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:18.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:18.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:18.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-24T18:22:18.933 INFO:tasks.workunit.client.0.trial172.stdout:6 2026-02-24T18:22:18.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:18.934 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:22:19.086 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:19.082+0000 7f70aa774a40 -1 osd.2 47 log_to_monitors true 2026-02-24T18:22:19.150 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:20.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:20.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:20.152 INFO:tasks.workunit.client.0.trial172.stdout:7 2026-02-24T18:22:20.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-24T18:22:20.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:20.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:22:20.372 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 58 up_thru 58 down_at 48 last_clean_interval [39,47) [v2:127.0.0.1:6818/1471319599,v1:127.0.0.1:6819/1471319599] [v2:127.0.0.1:6820/1471319599,v1:127.0.0.1:6821/1471319599] exists,up c77df6a7-2ace-45f0-9cdb-0238c4cea4fe 2026-02-24T18:22:20.372 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:22:20.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:22:20.373 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:22:20.373 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:226: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-24T18:22:20.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:226: TEST_bluestore: osd_pid2=120543 2026-02-24T18:22:20.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:227: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 3 2026-02-24T18:22:20.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:22:20.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:22:20.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-24T18:22:20.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:22:20.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-24T18:22:20.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:22:20.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:22:20.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:22:20.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:22:20.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-24T18:22:20.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-24T18:22:20.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:22:20.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:22:20.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:22:20.376 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:22:20.376 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:22:20.376 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:22:20.377 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:22:20.377 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:22:20.377 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:22:20.378 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:22:20.378 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:22:20.378 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:22:20.378 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:22:20.378 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:22:20.378 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:22:20.378 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:22:20.378 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:22:20.378 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:22:20.378 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:22:20.378 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:22:20.378 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-24T18:22:20.379 INFO:tasks.workunit.client.0.trial172.stdout:start osd.3 2026-02-24T18:22:20.379 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-24T18:22:20.379 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:22:20.379 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/3/whoami 2026-02-24T18:22:20.379 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-24T18:22:20.380 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:22:20.380 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:22:20.380 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:22:20.400 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:20.394+0000 7f3738738a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:22:20.401 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:20.394+0000 7f3738738a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:22:20.402 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:20.398+0000 7f3738738a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:22:20.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-24T18:22:20.598 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:22:20.598 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T18:22:20.598 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:22:20.598 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:22:20.598 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:20.598 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:22:20.599 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:22:20.600 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:20.600 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:22:20.816 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:20.924 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:20.918+0000 7f3738738a40 -1 Falling back to public interface 2026-02-24T18:22:21.817 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:21.817 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:21.818 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:22:21.818 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:22:21.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:21.819 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:22:22.028 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:23.029 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:23.029 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:23.029 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:22:23.030 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:22:23.030 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:23.030 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:22:23.239 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:24.241 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:24.241 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:24.241 INFO:tasks.workunit.client.0.trial172.stdout:3 2026-02-24T18:22:24.241 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-24T18:22:24.241 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:24.241 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:22:24.454 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:25.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:25.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:25.455 INFO:tasks.workunit.client.0.trial172.stdout:4 2026-02-24T18:22:25.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-24T18:22:25.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:25.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:22:25.664 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:26.666 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:26.666 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:26.666 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-24T18:22:26.667 INFO:tasks.workunit.client.0.trial172.stdout:5 2026-02-24T18:22:26.667 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:26.667 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:22:26.874 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:27.875 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:27.875 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:27.876 INFO:tasks.workunit.client.0.trial172.stdout:6 2026-02-24T18:22:27.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-24T18:22:27.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:27.877 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:22:28.092 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:22:28.263 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:22:28.258+0000 7f3738738a40 -1 osd.3 49 log_to_monitors true 2026-02-24T18:22:29.093 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:22:29.094 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:22:29.094 INFO:tasks.workunit.client.0.trial172.stdout:7 2026-02-24T18:22:29.095 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-24T18:22:29.095 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:22:29.095 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:22:29.313 INFO:tasks.workunit.client.0.trial172.stdout:osd.3 up in weight 1 up_from 61 up_thru 61 down_at 50 last_clean_interval [42,49) [v2:127.0.0.1:6826/2238966123,v1:127.0.0.1:6827/2238966123] [v2:127.0.0.1:6828/2238966123,v1:127.0.0.1:6829/2238966123] exists,up 79d5a161-1bb7-472e-8989-c86f625396ef 2026-02-24T18:22:29.313 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:22:29.314 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:22:29.314 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:22:29.314 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:228: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-24T18:22:29.314 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:228: TEST_bluestore: osd_pid3=121614 2026-02-24T18:22:29.314 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:231: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-24T18:22:29.332 INFO:tasks.workunit.client.0.trial172.stdout:hints = 1 2026-02-24T18:22:29.333 INFO:tasks.workunit.client.0.trial172.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-24T18:22:29.333 INFO:tasks.workunit.client.0.trial172.stdout:Object prefix: benchmark_data_trial172_122665 2026-02-24T18:22:29.333 INFO:tasks.workunit.client.0.trial172.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T18:22:29.333 INFO:tasks.workunit.client.0.trial172.stdout: 0 0 0 0 0 0 - 0 2026-02-24T18:22:30.333 INFO:tasks.workunit.client.0.trial172.stdout: 1 16 10954 10938 42.7234 42.7266 0.00142126 0.00146042 2026-02-24T18:22:31.333 INFO:tasks.workunit.client.0.trial172.stdout: 2 16 21646 21630 42.2429 41.7656 0.00241154 0.0014774 2026-02-24T18:22:32.333 INFO:tasks.workunit.client.0.trial172.stdout: 3 16 32286 32270 42.0151 41.5625 0.00148642 0.00148575 2026-02-24T18:22:33.333 INFO:tasks.workunit.client.0.trial172.stdout: 4 16 42862 42846 41.8386 41.3125 0.00170755 0.00149205 2026-02-24T18:22:34.333 INFO:tasks.workunit.client.0.trial172.stdout: 5 16 53535 53519 41.8083 41.6914 0.00125769 0.0014932 2026-02-24T18:22:35.333 INFO:tasks.workunit.client.0.trial172.stdout: 6 16 61891 61875 40.2799 32.6406 0.0743059 0.00153781 2026-02-24T18:22:36.333 INFO:tasks.workunit.client.0.trial172.stdout: 7 16 66127 66111 36.889 16.5469 0.00199341 0.00169258 2026-02-24T18:22:37.333 INFO:tasks.workunit.client.0.trial172.stdout: 8 16 76709 76693 37.4444 41.3359 0.00129698 0.00166754 2026-02-24T18:22:38.333 INFO:tasks.workunit.client.0.trial172.stdout: 9 16 87377 87361 37.9138 41.6719 0.00138109 0.00164691 2026-02-24T18:22:39.334 INFO:tasks.workunit.client.0.trial172.stdout: 10 16 97953 97937 38.2533 41.3125 0.00203692 0.0016323 2026-02-24T18:22:40.334 INFO:tasks.workunit.client.0.trial172.stdout: 11 16 108442 108426 38.5002 40.9727 0.00153573 0.00162184 2026-02-24T18:22:41.333 INFO:tasks.workunit.client.0.trial172.stdout: 12 16 118376 118360 38.5253 38.8047 0.000863104 0.00162057 2026-02-24T18:22:42.334 INFO:tasks.workunit.client.0.trial172.stdout: 13 16 128268 128252 38.534 38.6406 0.00280495 0.00162039 2026-02-24T18:22:43.334 INFO:tasks.workunit.client.0.trial172.stdout: 14 16 138285 138269 38.5763 39.1289 0.0021344 0.00161866 2026-02-24T18:22:44.334 INFO:tasks.workunit.client.0.trial172.stdout: 15 15 148332 148317 38.621 39.25 0.00251398 0.00161676 2026-02-24T18:22:45.334 INFO:tasks.workunit.client.0.trial172.stdout: 16 16 158141 158125 38.6016 38.3125 0.00134443 0.00161757 2026-02-24T18:22:46.334 INFO:tasks.workunit.client.0.trial172.stdout: 17 16 166137 166121 38.1681 31.2344 0.0849033 0.00163324 2026-02-24T18:22:47.334 INFO:tasks.workunit.client.0.trial172.stdout: 18 16 168005 167989 36.453 7.29688 0.00210895 0.00171301 2026-02-24T18:22:48.334 INFO:tasks.workunit.client.0.trial172.stdout: 19 16 177902 177886 36.569 38.6602 0.00225034 0.00170756 2026-02-24T18:22:49.334 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:22:49.333593+0000 min lat: 0.000471827 max lat: 0.180733 avg lat: 0.00170237 2026-02-24T18:22:49.334 INFO:tasks.workunit.client.0.trial172.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T18:22:49.334 INFO:tasks.workunit.client.0.trial172.stdout: 20 16 187839 187823 36.6812 38.8164 0.0012806 0.00170237 2026-02-24T18:22:50.334 INFO:tasks.workunit.client.0.trial172.stdout: 21 16 197745 197729 36.7769 38.6953 0.00175303 0.0016979 2026-02-24T18:22:51.334 INFO:tasks.workunit.client.0.trial172.stdout: 22 15 207513 207498 36.8397 38.1602 0.00148221 0.00169503 2026-02-24T18:22:52.334 INFO:tasks.workunit.client.0.trial172.stdout: 23 16 216702 216686 36.7983 35.8906 0.0332268 0.00169272 2026-02-24T18:22:53.335 INFO:tasks.workunit.client.0.trial172.stdout: 24 16 216769 216753 35.2759 0.261719 0.103705 0.0017617 2026-02-24T18:22:54.335 INFO:tasks.workunit.client.0.trial172.stdout: 25 16 217222 217206 33.9357 1.76953 0.00154875 0.00184022 2026-02-24T18:22:55.334 INFO:tasks.workunit.client.0.trial172.stdout: 26 16 227147 227131 34.1215 38.7695 0.00179291 0.00183019 2026-02-24T18:22:56.335 INFO:tasks.workunit.client.0.trial172.stdout: 27 16 237062 237046 34.2921 38.7305 0.00157166 0.0018211 2026-02-24T18:22:57.335 INFO:tasks.workunit.client.0.trial172.stdout: 28 16 246769 246753 34.4215 37.918 0.00135343 0.00181421 2026-02-24T18:22:58.335 INFO:tasks.workunit.client.0.trial172.stdout: 29 16 256295 256279 34.5176 37.2109 0.00189437 0.00180919 2026-02-24T18:22:59.335 INFO:tasks.workunit.client.0.trial172.stdout:Total time run: 30.0018 2026-02-24T18:22:59.335 INFO:tasks.workunit.client.0.trial172.stdout:Total writes made: 265946 2026-02-24T18:22:59.335 INFO:tasks.workunit.client.0.trial172.stdout:Write size: 4096 2026-02-24T18:22:59.335 INFO:tasks.workunit.client.0.trial172.stdout:Object size: 4096 2026-02-24T18:22:59.335 INFO:tasks.workunit.client.0.trial172.stdout:Bandwidth (MB/sec): 34.6263 2026-02-24T18:22:59.335 INFO:tasks.workunit.client.0.trial172.stdout:Stddev Bandwidth: 11.942 2026-02-24T18:22:59.336 INFO:tasks.workunit.client.0.trial172.stdout:Max bandwidth (MB/sec): 42.7266 2026-02-24T18:22:59.336 INFO:tasks.workunit.client.0.trial172.stdout:Min bandwidth (MB/sec): 0.261719 2026-02-24T18:22:59.336 INFO:tasks.workunit.client.0.trial172.stdout:Average IOPS: 8864 2026-02-24T18:22:59.336 INFO:tasks.workunit.client.0.trial172.stdout:Stddev IOPS: 3057.16 2026-02-24T18:22:59.336 INFO:tasks.workunit.client.0.trial172.stdout:Max IOPS: 10938 2026-02-24T18:22:59.336 INFO:tasks.workunit.client.0.trial172.stdout:Min IOPS: 67 2026-02-24T18:22:59.336 INFO:tasks.workunit.client.0.trial172.stdout:Average Latency(s): 0.00180349 2026-02-24T18:22:59.336 INFO:tasks.workunit.client.0.trial172.stdout:Stddev Latency(s): 0.00590051 2026-02-24T18:22:59.336 INFO:tasks.workunit.client.0.trial172.stdout:Max latency(s): 0.416357 2026-02-24T18:22:59.336 INFO:tasks.workunit.client.0.trial172.stdout:Min latency(s): 0.000471827 2026-02-24T18:22:59.338 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: kill 118401 2026-02-24T18:22:59.338 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: sleep 1 2026-02-24T18:23:00.340 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: kill 118401 2026-02-24T18:23:00.340 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 234: kill: (118401) - No such process 2026-02-24T18:23:00.340 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:235: TEST_bluestore: ceph osd down 0 2026-02-24T18:23:00.757 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already down. 2026-02-24T18:23:00.768 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: kill 119472 2026-02-24T18:23:00.768 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: sleep 1 2026-02-24T18:23:01.769 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: kill 119472 2026-02-24T18:23:01.770 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 236: kill: (119472) - No such process 2026-02-24T18:23:01.770 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:237: TEST_bluestore: ceph osd down 1 2026-02-24T18:23:01.983 INFO:tasks.workunit.client.0.trial172.stderr:osd.1 is already down. 2026-02-24T18:23:01.993 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: kill 120543 2026-02-24T18:23:01.993 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: sleep 1 2026-02-24T18:23:02.994 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: kill 120543 2026-02-24T18:23:02.995 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 238: kill: (120543) - No such process 2026-02-24T18:23:02.995 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:239: TEST_bluestore: ceph osd down 2 2026-02-24T18:23:03.202 INFO:tasks.workunit.client.0.trial172.stderr:osd.2 is already down. 2026-02-24T18:23:03.211 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:240: TEST_bluestore: kill 121614 2026-02-24T18:23:03.211 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:240: TEST_bluestore: sleep 1 2026-02-24T18:23:04.213 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:240: TEST_bluestore: kill 121614 2026-02-24T18:23:04.213 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 240: kill: (121614) - No such process 2026-02-24T18:23:04.214 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:241: TEST_bluestore: ceph osd down 3 2026-02-24T18:23:04.479 INFO:tasks.workunit.client.0.trial172.stderr:osd.3 is already down. 2026-02-24T18:23:04.488 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:244: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-24T18:23:09.498 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:23:09.561 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:246: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/0/db2 count=1024 bs=1M 2026-02-24T18:23:10.194 INFO:tasks.workunit.client.0.trial172.stderr:1024+0 records in 2026-02-24T18:23:10.194 INFO:tasks.workunit.client.0.trial172.stderr:1024+0 records out 2026-02-24T18:23:10.194 INFO:tasks.workunit.client.0.trial172.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.632656 s, 1.7 GB/s 2026-02-24T18:23:10.194 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:247: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.db --dev-target td/osd-bluefs-volume-ops/0/db2 --command bluefs-bdev-migrate 2026-02-24T18:23:10.211 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:23:15.158 INFO:tasks.workunit.client.0.trial172.stdout: device removed:1 td/osd-bluefs-volume-ops/0/block.db 2026-02-24T18:23:15.158 INFO:tasks.workunit.client.0.trial172.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/0/db2 2026-02-24T18:23:15.187 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:252: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-24T18:23:20.158 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:23:20.217 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:256: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/0/wal2 count=512 bs=1M 2026-02-24T18:23:20.531 INFO:tasks.workunit.client.0.trial172.stderr:512+0 records in 2026-02-24T18:23:20.531 INFO:tasks.workunit.client.0.trial172.stderr:512+0 records out 2026-02-24T18:23:20.531 INFO:tasks.workunit.client.0.trial172.stderr:536870912 bytes (537 MB, 512 MiB) copied, 0.31252 s, 1.7 GB/s 2026-02-24T18:23:20.531 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:257: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.wal --dev-target td/osd-bluefs-volume-ops/0/wal2 --command bluefs-bdev-migrate 2026-02-24T18:23:20.548 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:23:25.498 INFO:tasks.workunit.client.0.trial172.stdout: device removed:0 td/osd-bluefs-volume-ops/0/block.wal 2026-02-24T18:23:25.498 INFO:tasks.workunit.client.0.trial172.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/0/wal2 2026-02-24T18:23:25.531 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:261: TEST_bluestore: rm -rf td/osd-bluefs-volume-ops/0/wal 2026-02-24T18:23:25.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:263: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-24T18:23:30.570 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:23:30.631 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:266: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-24T18:23:35.530 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:23:35.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:268: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/1/db2 count=1024 bs=1M 2026-02-24T18:23:36.211 INFO:tasks.workunit.client.0.trial172.stderr:1024+0 records in 2026-02-24T18:23:36.211 INFO:tasks.workunit.client.0.trial172.stderr:1024+0 records out 2026-02-24T18:23:36.212 INFO:tasks.workunit.client.0.trial172.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.619679 s, 1.7 GB/s 2026-02-24T18:23:36.212 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:269: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block.db --devs-source td/osd-bluefs-volume-ops/1/block.wal --dev-target td/osd-bluefs-volume-ops/1/db2 --command bluefs-bdev-migrate 2026-02-24T18:23:36.228 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:23:41.578 INFO:tasks.workunit.client.0.trial172.stdout: device removed:1 td/osd-bluefs-volume-ops/1/block.db 2026-02-24T18:23:41.579 INFO:tasks.workunit.client.0.trial172.stdout: device removed:0 td/osd-bluefs-volume-ops/1/block.wal 2026-02-24T18:23:41.579 INFO:tasks.workunit.client.0.trial172.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/1/db2 2026-02-24T18:23:41.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:275: TEST_bluestore: rm -rf td/osd-bluefs-volume-ops/1/db 2026-02-24T18:23:41.739 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:277: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-24T18:23:46.626 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:23:46.680 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:279: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block.db --dev-target td/osd-bluefs-volume-ops/1/block --command bluefs-bdev-migrate 2026-02-24T18:23:46.697 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:23:51.146 INFO:tasks.workunit.client.0.trial172.stdout: device removed:1 td/osd-bluefs-volume-ops/1/block.db 2026-02-24T18:23:51.183 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:284: TEST_bluestore: rm -rf td/osd-bluefs-volume-ops/1/db2 2026-02-24T18:23:51.310 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:286: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-24T18:23:56.054 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:23:56.110 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:289: TEST_bluestore: ceph-objectstore-tool --type bluestore --data-path td/osd-bluefs-volume-ops/2 --op fsck --no-mon-config 2026-02-24T18:24:00.706 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:24:00.767 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:292: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/2/db2 count=1024 bs=1M 2026-02-24T18:24:01.400 INFO:tasks.workunit.client.0.trial172.stderr:1024+0 records in 2026-02-24T18:24:01.400 INFO:tasks.workunit.client.0.trial172.stderr:1024+0 records out 2026-02-24T18:24:01.400 INFO:tasks.workunit.client.0.trial172.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.631584 s, 1.7 GB/s 2026-02-24T18:24:01.400 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:293: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block --dev-target td/osd-bluefs-volume-ops/2/db2 --command bluefs-bdev-migrate 2026-02-24T18:24:01.417 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:24:01.538 INFO:tasks.workunit.client.0.trial172.stderr:Unable to migrate Slow volume to new location, please allocate new DB or WAL with --bluefs-bdev-new-db(wal) command 2026-02-24T18:24:01.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:299: TEST_bluestore: '[' 1 -eq 0 ']' 2026-02-24T18:24:01.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:302: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-24T18:24:06.094 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:24:06.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:305: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/2/db2 count=1024 bs=1M 2026-02-24T18:24:07.539 INFO:tasks.workunit.client.0.trial172.stderr:1024+0 records in 2026-02-24T18:24:07.539 INFO:tasks.workunit.client.0.trial172.stderr:1024+0 records out 2026-02-24T18:24:07.540 INFO:tasks.workunit.client.0.trial172.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.23816 s, 867 MB/s 2026-02-24T18:24:07.540 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:307: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block --devs-source td/osd-bluefs-volume-ops/2/block.db --devs-source td/osd-bluefs-volume-ops/2/block.wal --dev-target td/osd-bluefs-volume-ops/2/db2 --command bluefs-bdev-migrate 2026-02-24T18:24:07.555 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:24:11.874 INFO:tasks.workunit.client.0.trial172.stdout: device removed:1 td/osd-bluefs-volume-ops/2/block.db 2026-02-24T18:24:11.874 INFO:tasks.workunit.client.0.trial172.stdout: device removed:0 td/osd-bluefs-volume-ops/2/block.wal 2026-02-24T18:24:11.875 INFO:tasks.workunit.client.0.trial172.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/2/db2 2026-02-24T18:24:11.906 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:314: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-24T18:24:16.582 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:24:16.641 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:317: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/3/wal2 count=1024 bs=1M 2026-02-24T18:24:17.261 INFO:tasks.workunit.client.0.trial172.stderr:1024+0 records in 2026-02-24T18:24:17.261 INFO:tasks.workunit.client.0.trial172.stderr:1024+0 records out 2026-02-24T18:24:17.261 INFO:tasks.workunit.client.0.trial172.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.618953 s, 1.7 GB/s 2026-02-24T18:24:17.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:319: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block --devs-source td/osd-bluefs-volume-ops/3/block.wal --dev-target td/osd-bluefs-volume-ops/3/wal2 --command bluefs-bdev-migrate 2026-02-24T18:24:17.279 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:24:22.414 INFO:tasks.workunit.client.0.trial172.stdout: device removed:1 td/osd-bluefs-volume-ops/3/block 2026-02-24T18:24:22.414 INFO:tasks.workunit.client.0.trial172.stdout: device removed:0 td/osd-bluefs-volume-ops/3/block.wal 2026-02-24T18:24:22.415 INFO:tasks.workunit.client.0.trial172.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/3/wal2 2026-02-24T18:24:22.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:325: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-24T18:24:27.150 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:24:27.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:327: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 0 2026-02-24T18:24:27.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:24:27.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:24:27.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-24T18:24:27.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:24:27.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-24T18:24:27.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:24:27.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:24:27.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:24:27.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:24:27.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-24T18:24:27.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-24T18:24:27.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:24:27.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:24:27.201 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:24:27.202 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:24:27.202 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:24:27.202 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:24:27.202 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:24:27.202 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:24:27.202 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:24:27.202 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:24:27.202 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:24:27.202 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:24:27.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:24:27.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:24:27.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:24:27.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:24:27.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:24:27.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:24:27.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:24:27.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:24:27.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-24T18:24:27.204 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:24:27.204 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-24T18:24:27.204 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:24:27.204 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-24T18:24:27.206 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-24T18:24:27.206 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:24:27.206 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:24:27.206 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:24:27.227 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:24:27.224+0000 7fcf6c3c4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:24:27.228 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:24:27.224+0000 7fcf6c3c4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:24:27.230 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:24:27.224+0000 7fcf6c3c4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:24:27.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-24T18:24:27.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:24:27.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:24:27.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:24:27.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:24:27.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:27.423 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:24:27.423 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:24:27.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:27.424 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:24:27.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:27.727 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:24:27.724+0000 7fcf6c3c4a40 -1 Falling back to public interface 2026-02-24T18:24:28.646 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:28.647 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:28.647 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:24:28.648 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:24:28.648 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:28.648 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:24:28.867 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:29.869 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:29.869 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:29.870 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:24:29.870 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:24:29.871 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:29.871 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:24:30.088 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:31.089 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:31.089 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:31.090 INFO:tasks.workunit.client.0.trial172.stdout:3 2026-02-24T18:24:31.090 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-24T18:24:31.091 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:31.091 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:24:31.309 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:32.311 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:32.311 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:32.311 INFO:tasks.workunit.client.0.trial172.stdout:4 2026-02-24T18:24:32.312 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-24T18:24:32.312 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:32.312 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:24:32.525 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:33.526 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:33.526 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:33.527 INFO:tasks.workunit.client.0.trial172.stdout:5 2026-02-24T18:24:33.527 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-24T18:24:33.527 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:33.527 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:24:33.745 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:34.746 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:34.746 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:34.746 INFO:tasks.workunit.client.0.trial172.stdout:6 2026-02-24T18:24:34.746 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-24T18:24:34.746 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:34.747 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:24:34.970 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:35.971 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:35.971 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:35.971 INFO:tasks.workunit.client.0.trial172.stdout:7 2026-02-24T18:24:35.971 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-24T18:24:35.971 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:35.971 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:24:36.190 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:37.191 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:37.192 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:37.192 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2026-02-24T18:24:37.192 INFO:tasks.workunit.client.0.trial172.stdout:8 2026-02-24T18:24:37.193 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:37.193 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:24:37.380 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:24:37.376+0000 7fcf6c3c4a40 -1 osd.0 63 log_to_monitors true 2026-02-24T18:24:37.419 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:38.420 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:38.421 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:38.421 INFO:tasks.workunit.client.0.trial172.stdout:9 2026-02-24T18:24:38.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2026-02-24T18:24:38.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:38.422 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:24:38.632 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 71 up_thru 71 down_at 63 last_clean_interval [52,63) [v2:127.0.0.1:6802/4219031864,v1:127.0.0.1:6803/4219031864] [v2:127.0.0.1:6804/4219031864,v1:127.0.0.1:6805/4219031864] exists,up 410d8f94-9514-413c-9295-e8e271a743d8 2026-02-24T18:24:38.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:24:38.633 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:24:38.634 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:24:38.634 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:328: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-24T18:24:38.637 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:328: TEST_bluestore: osd_pid0=128840 2026-02-24T18:24:38.637 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:329: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 1 2026-02-24T18:24:38.637 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:24:38.637 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:24:38.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T18:24:38.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:24:38.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-24T18:24:38.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:24:38.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:24:38.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:24:38.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:24:38.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-24T18:24:38.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-24T18:24:38.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:24:38.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:24:38.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:24:38.639 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:24:38.639 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:24:38.639 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:24:38.639 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:24:38.639 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:24:38.639 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:24:38.639 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:24:38.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:24:38.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:24:38.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:24:38.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:24:38.639 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:24:38.640 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:24:38.640 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:24:38.640 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:24:38.640 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:24:38.640 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:24:38.640 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-24T18:24:38.641 INFO:tasks.workunit.client.0.trial172.stdout:start osd.1 2026-02-24T18:24:38.642 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T18:24:38.642 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:24:38.642 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/1/whoami 2026-02-24T18:24:38.642 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T18:24:38.642 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:24:38.642 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:24:38.642 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:24:38.660 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:24:38.656+0000 7fe49c037a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:24:38.661 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:24:38.656+0000 7fe49c037a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:24:38.662 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:24:38.660+0000 7fe49c037a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:24:38.850 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T18:24:38.850 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:24:38.851 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T18:24:38.851 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:24:38.851 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:24:38.851 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:38.851 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:24:38.852 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:24:38.853 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:38.853 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:24:39.047 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:24:39.044+0000 7fe49c037a40 -1 Falling back to public interface 2026-02-24T18:24:39.071 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:40.073 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:40.073 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:40.074 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:24:40.075 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:24:40.075 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:40.075 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:24:40.288 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:41.289 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:41.289 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:41.289 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:24:41.289 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:24:41.289 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:41.289 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:24:41.499 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:42.500 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:42.500 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:42.501 INFO:tasks.workunit.client.0.trial172.stdout:3 2026-02-24T18:24:42.501 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-24T18:24:42.501 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:42.501 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:24:42.716 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:43.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:43.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:43.718 INFO:tasks.workunit.client.0.trial172.stdout:4 2026-02-24T18:24:43.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-24T18:24:43.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:43.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:24:43.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:44.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:44.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:44.933 INFO:tasks.workunit.client.0.trial172.stdout:5 2026-02-24T18:24:44.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-24T18:24:44.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:44.933 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:24:45.144 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:46.146 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:46.146 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:46.146 INFO:tasks.workunit.client.0.trial172.stdout:6 2026-02-24T18:24:46.147 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-24T18:24:46.147 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:46.147 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:24:46.357 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:47.358 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:47.358 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:47.358 INFO:tasks.workunit.client.0.trial172.stdout:7 2026-02-24T18:24:47.358 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-24T18:24:47.359 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:47.359 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:24:47.572 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:48.573 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:48.573 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:48.573 INFO:tasks.workunit.client.0.trial172.stdout:8 2026-02-24T18:24:48.574 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2026-02-24T18:24:48.574 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:48.574 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:24:48.666 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:24:48.664+0000 7fe49c037a40 -1 osd.1 64 log_to_monitors true 2026-02-24T18:24:48.799 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:49.800 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:49.801 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:49.801 INFO:tasks.workunit.client.0.trial172.stdout:9 2026-02-24T18:24:49.801 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2026-02-24T18:24:49.801 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:49.801 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T18:24:50.012 INFO:tasks.workunit.client.0.trial172.stdout:osd.1 up in weight 1 up_from 74 up_thru 74 down_at 65 last_clean_interval [55,64) [v2:127.0.0.1:6810/509781717,v1:127.0.0.1:6811/509781717] [v2:127.0.0.1:6812/509781717,v1:127.0.0.1:6813/509781717] exists,up f4b079dc-13d9-4358-9502-d9202bc6b891 2026-02-24T18:24:50.013 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:24:50.013 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:24:50.013 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:24:50.013 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:330: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-24T18:24:50.015 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:330: TEST_bluestore: osd_pid1=129962 2026-02-24T18:24:50.016 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:331: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 2 2026-02-24T18:24:50.016 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:24:50.016 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:24:50.016 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T18:24:50.016 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:24:50.016 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-24T18:24:50.016 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:24:50.016 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:24:50.016 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:24:50.016 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:24:50.017 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-24T18:24:50.017 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-24T18:24:50.017 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:24:50.017 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:24:50.017 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:24:50.017 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:24:50.017 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:24:50.017 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:24:50.017 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:24:50.017 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:24:50.017 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:24:50.019 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:24:50.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:24:50.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:24:50.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:24:50.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:24:50.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:24:50.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:24:50.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:24:50.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:24:50.020 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:24:50.020 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:24:50.020 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-24T18:24:50.020 INFO:tasks.workunit.client.0.trial172.stdout:start osd.2 2026-02-24T18:24:50.021 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T18:24:50.021 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:24:50.021 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/2/whoami 2026-02-24T18:24:50.021 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T18:24:50.021 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:24:50.021 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:24:50.022 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:24:50.035 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:24:50.032+0000 7fe540516a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:24:50.037 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:24:50.032+0000 7fe540516a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:24:50.038 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:24:50.036+0000 7fe540516a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:24:50.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T18:24:50.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:24:50.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T18:24:50.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:24:50.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:24:50.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:50.236 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:24:50.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:24:50.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:50.236 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:24:50.454 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:50.544 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:24:50.540+0000 7fe540516a40 -1 Falling back to public interface 2026-02-24T18:24:51.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:51.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:51.455 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:24:51.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:24:51.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:51.455 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:24:51.677 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:52.679 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:52.679 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:52.679 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:24:52.680 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:24:52.680 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:52.680 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:24:52.898 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:53.899 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:53.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:53.900 INFO:tasks.workunit.client.0.trial172.stdout:3 2026-02-24T18:24:53.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-24T18:24:53.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:53.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:24:54.123 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:55.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:55.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:55.124 INFO:tasks.workunit.client.0.trial172.stdout:4 2026-02-24T18:24:55.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-24T18:24:55.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:55.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:24:55.339 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:56.340 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:56.341 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:56.341 INFO:tasks.workunit.client.0.trial172.stdout:5 2026-02-24T18:24:56.341 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-24T18:24:56.341 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:56.341 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:24:56.559 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:57.560 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:57.560 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:57.560 INFO:tasks.workunit.client.0.trial172.stdout:6 2026-02-24T18:24:57.561 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-24T18:24:57.561 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:57.561 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:24:57.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:58.776 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:24:58.776 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:24:58.777 INFO:tasks.workunit.client.0.trial172.stdout:7 2026-02-24T18:24:58.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-24T18:24:58.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:24:58.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:24:59.002 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:24:59.539 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:24:59.536+0000 7fe540516a40 -1 osd.2 66 log_to_monitors true 2026-02-24T18:25:00.003 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:25:00.004 INFO:tasks.workunit.client.0.trial172.stdout:8 2026-02-24T18:25:00.004 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:25:00.004 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2026-02-24T18:25:00.004 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:25:00.004 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T18:25:00.217 INFO:tasks.workunit.client.0.trial172.stdout:osd.2 up in weight 1 up_from 78 up_thru 78 down_at 67 last_clean_interval [58,66) [v2:127.0.0.1:6818/2862097256,v1:127.0.0.1:6819/2862097256] [v2:127.0.0.1:6820/2862097256,v1:127.0.0.1:6821/2862097256] exists,up c77df6a7-2ace-45f0-9cdb-0238c4cea4fe 2026-02-24T18:25:00.218 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:25:00.218 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:25:00.218 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:25:00.219 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:332: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-24T18:25:00.222 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:332: TEST_bluestore: osd_pid2=131073 2026-02-24T18:25:00.222 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:333: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 3 2026-02-24T18:25:00.222 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:25:00.222 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:25:00.222 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-24T18:25:00.222 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:25:00.223 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-24T18:25:00.223 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 ' 2026-02-24T18:25:00.223 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:25:00.223 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:25:00.223 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:25:00.223 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-24T18:25:00.223 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-24T18:25:00.223 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:25:00.223 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:25:00.223 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:25:00.223 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:25:00.223 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:25:00.224 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:25:00.224 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:25:00.224 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:25:00.224 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:25:00.224 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:25:00.224 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:25:00.224 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:25:00.224 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:25:00.224 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:25:00.224 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:25:00.224 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:25:00.224 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:25:00.225 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:25:00.225 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:25:00.225 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:25:00.225 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-24T18:25:00.225 INFO:tasks.workunit.client.0.trial172.stdout:start osd.3 2026-02-24T18:25:00.226 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-24T18:25:00.226 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:25:00.226 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/3/whoami 2026-02-24T18:25:00.227 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-24T18:25:00.227 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:25:00.227 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:25:00.227 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:25:00.241 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:25:00.236+0000 7f6161abba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:25:00.243 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:25:00.240+0000 7f6161abba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:25:00.243 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:25:00.240+0000 7f6161abba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:25:00.436 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-24T18:25:00.436 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:25:00.437 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T18:25:00.437 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:25:00.437 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:25:00.437 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:25:00.437 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:25:00.437 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:25:00.437 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:25:00.437 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:25:00.656 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:25:00.748 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:25:00.744+0000 7f6161abba40 -1 Falling back to public interface 2026-02-24T18:25:01.657 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:25:01.657 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:25:01.658 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:25:01.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:25:01.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:25:01.658 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:25:01.875 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:25:02.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:25:02.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:25:02.876 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:25:02.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:25:02.876 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:25:02.877 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:25:03.099 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:25:04.100 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:25:04.101 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:25:04.101 INFO:tasks.workunit.client.0.trial172.stdout:3 2026-02-24T18:25:04.101 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-24T18:25:04.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:25:04.102 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:25:04.323 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:25:05.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:25:05.325 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:25:05.325 INFO:tasks.workunit.client.0.trial172.stdout:4 2026-02-24T18:25:05.325 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-24T18:25:05.325 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:25:05.325 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:25:05.545 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:25:06.546 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:25:06.547 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:25:06.547 INFO:tasks.workunit.client.0.trial172.stdout:5 2026-02-24T18:25:06.548 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-24T18:25:06.549 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:25:06.549 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:25:06.763 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:25:07.765 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:25:07.765 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:25:07.765 INFO:tasks.workunit.client.0.trial172.stdout:6 2026-02-24T18:25:07.766 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-24T18:25:07.766 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:25:07.766 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:25:07.985 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:25:08.986 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:25:08.986 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:25:08.987 INFO:tasks.workunit.client.0.trial172.stdout:7 2026-02-24T18:25:08.987 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-24T18:25:08.987 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:25:08.987 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:25:09.209 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:25:09.482 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:25:09.479+0000 7f6161abba40 -1 osd.3 68 log_to_monitors true 2026-02-24T18:25:10.210 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:25:10.210 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:25:10.210 INFO:tasks.workunit.client.0.trial172.stdout:8 2026-02-24T18:25:10.210 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2026-02-24T18:25:10.210 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:25:10.210 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T18:25:10.424 INFO:tasks.workunit.client.0.trial172.stdout:osd.3 up in weight 1 up_from 81 up_thru 81 down_at 69 last_clean_interval [61,68) [v2:127.0.0.1:6826/2039301456,v1:127.0.0.1:6827/2039301456] [v2:127.0.0.1:6828/2039301456,v1:127.0.0.1:6829/2039301456] exists,up 79d5a161-1bb7-472e-8989-c86f625396ef 2026-02-24T18:25:10.425 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:25:10.425 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:25:10.425 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:25:10.425 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:334: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-24T18:25:10.426 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:334: TEST_bluestore: osd_pid3=132164 2026-02-24T18:25:10.426 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:337: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-24T18:25:10.445 INFO:tasks.workunit.client.0.trial172.stdout:hints = 1 2026-02-24T18:25:10.446 INFO:tasks.workunit.client.0.trial172.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-24T18:25:10.446 INFO:tasks.workunit.client.0.trial172.stdout:Object prefix: benchmark_data_trial172_133243 2026-02-24T18:25:10.446 INFO:tasks.workunit.client.0.trial172.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T18:25:10.446 INFO:tasks.workunit.client.0.trial172.stdout: 0 0 0 0 0 0 - 0 2026-02-24T18:25:11.446 INFO:tasks.workunit.client.0.trial172.stdout: 1 16 9853 9837 38.4224 38.4258 0.00120179 0.00162371 2026-02-24T18:25:12.446 INFO:tasks.workunit.client.0.trial172.stdout: 2 16 20630 20614 40.2584 42.0977 0.00107262 0.00155031 2026-02-24T18:25:13.446 INFO:tasks.workunit.client.0.trial172.stdout: 3 16 31315 31299 40.7506 41.7383 0.00103778 0.00153184 2026-02-24T18:25:14.446 INFO:tasks.workunit.client.0.trial172.stdout: 4 16 41972 41956 40.9694 41.6289 0.000535865 0.00152299 2026-02-24T18:25:15.446 INFO:tasks.workunit.client.0.trial172.stdout: 5 16 52748 52732 41.1936 42.0938 0.00128766 0.00151461 2026-02-24T18:25:16.446 INFO:tasks.workunit.client.0.trial172.stdout: 6 16 63362 63346 41.2378 41.4609 0.00145665 0.00151396 2026-02-24T18:25:17.446 INFO:tasks.workunit.client.0.trial172.stdout: 7 16 74054 74038 41.3128 41.7656 0.00120181 0.00151126 2026-02-24T18:25:18.446 INFO:tasks.workunit.client.0.trial172.stdout: 8 16 84776 84760 41.3836 41.8828 0.00108316 0.00150869 2026-02-24T18:25:19.446 INFO:tasks.workunit.client.0.trial172.stdout: 9 16 95469 95453 41.4262 41.7695 0.000967596 0.00150713 2026-02-24T18:25:20.447 INFO:tasks.workunit.client.0.trial172.stdout: 10 16 106019 106003 41.4043 41.2109 0.00135252 0.00150795 2026-02-24T18:25:21.446 INFO:tasks.workunit.client.0.trial172.stdout: 11 16 116550 116534 41.3797 41.1367 0.00200512 0.00150888 2026-02-24T18:25:22.447 INFO:tasks.workunit.client.0.trial172.stdout: 12 16 120302 120286 39.1526 14.6562 0.0228743 0.00159246 2026-02-24T18:25:23.447 INFO:tasks.workunit.client.0.trial172.stdout: 13 16 129399 129383 38.8741 35.5352 0.00156361 0.00160623 2026-02-24T18:25:24.447 INFO:tasks.workunit.client.0.trial172.stdout: 14 16 140081 140065 39.0776 41.7266 0.00294582 0.00159777 2026-02-24T18:25:25.447 INFO:tasks.workunit.client.0.trial172.stdout: 15 16 150542 150526 39.1965 40.8633 0.00108427 0.001593 2026-02-24T18:25:26.447 INFO:tasks.workunit.client.0.trial172.stdout: 16 16 161107 161091 39.3258 41.2695 0.0011559 0.00158776 2026-02-24T18:25:27.447 INFO:tasks.workunit.client.0.trial172.stdout: 17 15 171339 171324 39.3636 39.9727 0.00174411 0.00158623 2026-02-24T18:25:28.447 INFO:tasks.workunit.client.0.trial172.stdout: 18 15 181189 181174 39.3142 38.4766 0.00145148 0.00158823 2026-02-24T18:25:29.447 INFO:tasks.workunit.client.0.trial172.stdout: 19 16 191331 191315 39.3298 39.6133 0.00146511 0.00158762 2026-02-24T18:25:30.447 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:25:30.446563+0000 min lat: 0.000514054 max lat: 0.0690945 avg lat: 0.00158703 2026-02-24T18:25:30.447 INFO:tasks.workunit.client.0.trial172.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T18:25:30.447 INFO:tasks.workunit.client.0.trial172.stdout: 20 16 201476 201460 39.3446 39.6289 0.0014981 0.00158703 2026-02-24T18:25:31.447 INFO:tasks.workunit.client.0.trial172.stdout: 21 16 211741 211725 39.3803 40.0977 0.000663732 0.00158546 2026-02-24T18:25:32.447 INFO:tasks.workunit.client.0.trial172.stdout: 22 16 221786 221770 39.3737 39.2383 0.00160518 0.00158586 2026-02-24T18:25:33.448 INFO:tasks.workunit.client.0.trial172.stdout: 23 16 225503 225487 38.293 14.5195 0.0218408 0.00162814 2026-02-24T18:25:34.447 INFO:tasks.workunit.client.0.trial172.stdout: 24 16 225658 225642 36.7227 0.605469 0.123056 0.00169179 2026-02-24T18:25:35.448 INFO:tasks.workunit.client.0.trial172.stdout: 25 16 225837 225821 35.2818 0.699219 0.0577927 0.00176721 2026-02-24T18:25:36.448 INFO:tasks.workunit.client.0.trial172.stdout: 26 16 230216 230200 34.5827 17.1055 0.0018244 0.00180576 2026-02-24T18:25:37.448 INFO:tasks.workunit.client.0.trial172.stdout: 27 16 240342 240326 34.7667 39.5547 0.00109992 0.0017962 2026-02-24T18:25:38.448 INFO:tasks.workunit.client.0.trial172.stdout: 28 16 250539 250523 34.9475 39.832 0.00173429 0.00178689 2026-02-24T18:25:39.448 INFO:tasks.workunit.client.0.trial172.stdout: 29 16 260784 260768 35.1223 40.0195 0.00139114 0.00177799 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Total time run: 30.0011 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Total writes made: 270874 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Write size: 4096 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Object size: 4096 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Bandwidth (MB/sec): 35.2688 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Stddev Bandwidth: 12.3683 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Max bandwidth (MB/sec): 42.0977 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Min bandwidth (MB/sec): 0.605469 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Average IOPS: 9028 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Stddev IOPS: 3166.29 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Max IOPS: 10777 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Min IOPS: 155 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Average Latency(s): 0.00177064 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Stddev Latency(s): 0.00454646 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Max latency(s): 0.191173 2026-02-24T18:25:40.448 INFO:tasks.workunit.client.0.trial172.stdout:Min latency(s): 0.000514054 2026-02-24T18:25:40.451 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:339: TEST_bluestore: wait_for_clean 2026-02-24T18:25:40.451 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:25:40.451 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:25:40.451 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:25:40.452 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T18:25:40.452 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:25:40.452 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:25:40.452 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:25:40.452 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:25:40.452 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:25:40.551 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T18:25:40.552 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:25:40.552 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:25:40.552 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:25:40.552 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:25:40.552 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:25:40.771 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T18:25:40.771 INFO:tasks.workunit.client.0.trial172.stderr:1 2026-02-24T18:25:40.771 INFO:tasks.workunit.client.0.trial172.stderr:2 2026-02-24T18:25:40.771 INFO:tasks.workunit.client.0.trial172.stderr:3' 2026-02-24T18:25:40.771 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:25:40.771 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:25:40.772 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:25:40.854 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=304942678031 2026-02-24T18:25:40.854 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 304942678031 2026-02-24T18:25:40.854 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-304942678031' 2026-02-24T18:25:40.855 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:25:40.855 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T18:25:40.935 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=317827579917 2026-02-24T18:25:40.935 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 317827579917 2026-02-24T18:25:40.935 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-304942678031 1-317827579917' 2026-02-24T18:25:40.935 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:25:40.936 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T18:25:41.012 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=335007449099 2026-02-24T18:25:41.012 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 335007449099 2026-02-24T18:25:41.012 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-304942678031 1-317827579917 2-335007449099' 2026-02-24T18:25:41.012 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:25:41.012 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T18:25:41.091 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=347892350985 2026-02-24T18:25:41.091 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 347892350985 2026-02-24T18:25:41.092 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-304942678031 1-317827579917 2-335007449099 3-347892350985' 2026-02-24T18:25:41.092 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:25:41.092 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-304942678031 2026-02-24T18:25:41.092 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:25:41.093 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:25:41.094 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-304942678031 2026-02-24T18:25:41.094 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:25:41.095 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=304942678031 2026-02-24T18:25:41.095 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 304942678031' 2026-02-24T18:25:41.095 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 304942678031 2026-02-24T18:25:41.096 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:25:41.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 304942678031 -lt 304942678031 2026-02-24T18:25:41.324 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:25:41.326 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-317827579917 2026-02-24T18:25:41.326 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:25:41.326 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T18:25:41.327 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-317827579917 2026-02-24T18:25:41.327 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:25:41.329 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=317827579917 2026-02-24T18:25:41.329 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 317827579917' 2026-02-24T18:25:41.329 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.1 seq 317827579917 2026-02-24T18:25:41.329 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T18:25:41.556 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 317827579917 -lt 317827579917 2026-02-24T18:25:41.556 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:25:41.557 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-335007449099 2026-02-24T18:25:41.557 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:25:41.558 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T18:25:41.559 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-335007449099 2026-02-24T18:25:41.559 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:25:41.561 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=335007449099 2026-02-24T18:25:41.561 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.2 seq 335007449099 2026-02-24T18:25:41.561 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 335007449099' 2026-02-24T18:25:41.561 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T18:25:41.786 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 335007449099 -lt 335007449099 2026-02-24T18:25:41.786 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:25:41.787 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-347892350985 2026-02-24T18:25:41.787 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:25:41.788 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T18:25:41.789 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-347892350985 2026-02-24T18:25:41.789 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:25:41.791 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=347892350985 2026-02-24T18:25:41.791 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 347892350985' 2026-02-24T18:25:41.791 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.3 seq 347892350985 2026-02-24T18:25:41.791 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T18:25:42.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 347892350983 -lt 347892350985 2026-02-24T18:25:42.006 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:25:43.007 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:25:43.008 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T18:25:43.228 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 347892350985 -lt 347892350985 2026-02-24T18:25:43.229 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:25:43.230 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:25:43.230 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:25:43.520 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-24T18:25:43.520 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:25:43.521 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:25:43.521 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:25:43.521 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:25:43.521 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:25:43.521 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:25:43.521 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:25:43.746 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-24T18:25:43.746 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:25:43.747 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:25:43.747 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:25:44.034 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-24T18:25:44.034 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T18:25:44.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T18:25:44.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:15: run: teardown td/osd-bluefs-volume-ops 2026-02-24T18:25:44.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:25:44.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:25:44.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-24T18:25:44.035 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:25:44.035 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:25:44.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:25:44.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:25:44.035 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:25:44.346 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:25:44.346 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:25:44.347 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:25:44.347 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:25:44.348 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:25:44.348 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:25:44.348 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:25:44.350 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:25:44.350 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:25:44.350 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:25:44.350 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:25:44.351 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:25:44.353 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:25:44.353 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-24T18:25:50.589 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:25:50.589 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:25:50.589 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:25:50.589 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.88188 2026-02-24T18:25:50.590 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:25:50.590 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:25:50.590 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:12: run: for func in $funcs 2026-02-24T18:25:50.590 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:13: run: setup td/osd-bluefs-volume-ops 2026-02-24T18:25:50.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:25:50.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-bluefs-volume-ops 2026-02-24T18:25:50.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:25:50.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:25:50.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-24T18:25:50.591 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:25:50.591 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:25:50.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:25:50.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:25:50.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:25:50.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:25:50.594 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:25:50.595 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:25:50.596 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:25:50.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:25:50.597 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:25:50.597 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:25:50.598 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:25:50.599 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:25:50.599 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:25:50.599 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:25:50.600 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:25:50.601 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:25:50.601 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-24T18:25:50.603 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:25:50.603 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:25:50.603 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:25:50.603 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.88188 2026-02-24T18:25:50.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:25:50.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:25:50.604 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-bluefs-volume-ops 2026-02-24T18:25:50.606 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T18:25:50.606 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:25:50.606 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:25:50.606 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.88188 2026-02-24T18:25:50.608 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T18:25:50.609 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T18:25:50.609 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T18:25:50.609 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-bluefs-volume-ops 1' TERM HUP INT 2026-02-24T18:25:50.609 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: TEST_bluestore2 td/osd-bluefs-volume-ops 2026-02-24T18:25:50.609 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:343: TEST_bluestore2: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:25:50.610 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:345: TEST_bluestore2: ulimit -n 2026-02-24T18:25:50.610 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:345: TEST_bluestore2: local flimit=4096 2026-02-24T18:25:50.610 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:346: TEST_bluestore2: '[' 4096 -lt 1536 ']' 2026-02-24T18:25:50.610 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:349: TEST_bluestore2: export CEPH_MON=127.0.0.1:7146 2026-02-24T18:25:50.610 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:349: TEST_bluestore2: CEPH_MON=127.0.0.1:7146 2026-02-24T18:25:50.610 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:350: TEST_bluestore2: export CEPH_ARGS 2026-02-24T18:25:50.610 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:351: TEST_bluestore2: uuidgen 2026-02-24T18:25:50.611 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:351: TEST_bluestore2: CEPH_ARGS+='--fsid=981d3548-b091-474d-b724-cb00f7e35350 --auth-supported=none ' 2026-02-24T18:25:50.611 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:352: TEST_bluestore2: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-24T18:25:50.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:353: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_size=4294967296 ' 2026-02-24T18:25:50.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:354: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_db_create=true ' 2026-02-24T18:25:50.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:355: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_db_size=1073741824 ' 2026-02-24T18:25:50.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:356: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_wal_create=false ' 2026-02-24T18:25:50.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:357: TEST_bluestore2: CEPH_ARGS+='--bluestore_fsck_on_mount=true ' 2026-02-24T18:25:50.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:358: TEST_bluestore2: CEPH_ARGS+='--osd_pool_default_size=1 ' 2026-02-24T18:25:50.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:359: TEST_bluestore2: CEPH_ARGS+='--osd_pool_default_min_size=1 ' 2026-02-24T18:25:50.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:360: TEST_bluestore2: CEPH_ARGS+='--bluestore_debug_enforce_settings=ssd ' 2026-02-24T18:25:50.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:362: TEST_bluestore2: run_mon td/osd-bluefs-volume-ops a 2026-02-24T18:25:50.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:25:50.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T18:25:50.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T18:25:50.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T18:25:50.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-bluefs-volume-ops/a 2026-02-24T18:25:50.612 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bluefs-volume-ops/a --run-dir=td/osd-bluefs-volume-ops 2026-02-24T18:25:50.637 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T18:25:50.637 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:25:50.637 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:25:50.638 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:25:50.638 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:25:50.638 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:25:50.638 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:25:50.638 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bluefs-volume-ops/a '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bluefs-volume-ops/log --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T18:25:50.668 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T18:25:50.669 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T18:25:50.669 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:25:50.669 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:25:50.669 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T18:25:50.670 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T18:25:50.671 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:25:50.671 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:25:50.671 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:25:50.671 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:25:50.671 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:25:50.671 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:25:50.671 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.88188/ceph-mon.a.asok 2026-02-24T18:25:50.671 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:25:50.671 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.88188/ceph-mon.a.asok config get fsid 2026-02-24T18:25:50.727 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T18:25:50.727 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:25:50.727 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:25:50.727 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T18:25:50.727 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T18:25:50.728 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:25:50.728 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:25:50.728 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:25:50.728 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:25:50.728 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:25:50.728 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:25:50.729 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.88188/ceph-mon.a.asok 2026-02-24T18:25:50.729 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:25:50.729 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.88188/ceph-mon.a.asok config get mon_host 2026-02-24T18:25:50.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:363: TEST_bluestore2: run_mgr td/osd-bluefs-volume-ops x 2026-02-24T18:25:50.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:25:50.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T18:25:50.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T18:25:50.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T18:25:50.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-bluefs-volume-ops/x 2026-02-24T18:25:50.787 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T18:25:50.903 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T18:25:50.903 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:25:50.903 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:25:50.903 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:25:50.903 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:25:50.903 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:25:50.903 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:25:50.904 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:25:50.905 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bluefs-volume-ops/x '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:25:50.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:364: TEST_bluestore2: run_osd td/osd-bluefs-volume-ops 0 2026-02-24T18:25:50.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:25:50.929 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:25:50.929 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T18:25:50.929 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:25:50.929 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-24T18:25:50.929 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=981d3548-b091-474d-b724-cb00f7e35350 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-24T18:25:50.929 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:25:50.929 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:25:50.929 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:25:50.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-24T18:25:50.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-24T18:25:50.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:25:50.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:25:50.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:25:50.930 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:25:50.930 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:25:50.930 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:25:50.930 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:25:50.930 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:25:50.930 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:25:50.930 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:25:50.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:25:50.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:25:50.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:25:50.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:25:50.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:25:50.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:25:50.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:25:50.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:25:50.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:25:50.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:25:50.931 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:25:50.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:25:50.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-24T18:25:50.932 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:25:50.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7bbcb87b-5704-449d-b1bd-4831165eb2bb 2026-02-24T18:25:50.932 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 7bbcb87b-5704-449d-b1bd-4831165eb2bb' 2026-02-24T18:25:50.933 INFO:tasks.workunit.client.0.trial172.stdout:add osd0 7bbcb87b-5704-449d-b1bd-4831165eb2bb 2026-02-24T18:25:50.933 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:25:50.943 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAu7Z1pfPQeOBAAVHmrzBJjCp3TUWwIuHXjIw== 2026-02-24T18:25:50.943 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAu7Z1pfPQeOBAAVHmrzBJjCp3TUWwIuHXjIw=="}' 2026-02-24T18:25:50.943 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7bbcb87b-5704-449d-b1bd-4831165eb2bb -i td/osd-bluefs-volume-ops/0/new.json 2026-02-24T18:25:51.060 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:25:51.068 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-bluefs-volume-ops/0/new.json 2026-02-24T18:25:51.069 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=981d3548-b091-474d-b724-cb00f7e35350 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAu7Z1pfPQeOBAAVHmrzBJjCp3TUWwIuHXjIw== --osd-uuid 7bbcb87b-5704-449d-b1bd-4831165eb2bb 2026-02-24T18:25:51.086 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:25:51.083+0000 7f3563b1fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:25:51.088 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:25:51.083+0000 7f3563b1fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:25:51.089 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:25:51.083+0000 7f3563b1fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:25:51.089 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:25:51.083+0000 7f3563b1fa40 -1 bdev(0x5650b41f6800 td/osd-bluefs-volume-ops/0/block) open stat got: (1) Operation not permitted 2026-02-24T18:25:51.089 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:25:51.083+0000 7f3563b1fa40 -1 bluestore(td/osd-bluefs-volume-ops/0) _read_fsid unparsable uuid 2026-02-24T18:25:52.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-bluefs-volume-ops/0/keyring 2026-02-24T18:25:52.645 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:25:52.646 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T18:25:52.646 INFO:tasks.workunit.client.0.trial172.stdout:adding osd0 key to auth repository 2026-02-24T18:25:52.646 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-bluefs-volume-ops/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:25:52.776 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:25:52.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T18:25:52.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=981d3548-b091-474d-b724-cb00f7e35350 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:25:52.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:25:52.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:25:52.777 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:25:52.804 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:25:52.799+0000 7f4239e6ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:25:52.806 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:25:52.799+0000 7f4239e6ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:25:52.807 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:25:52.803+0000 7f4239e6ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:25:52.892 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T18:25:52.893 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:25:52.893 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:25:52.893 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:25:52.893 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:25:52.893 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:25:52.893 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:25:52.894 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:25:52.894 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:25:52.894 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:25:53.009 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:25:53.376 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:25:53.371+0000 7f4239e6ba40 -1 Falling back to public interface 2026-02-24T18:25:54.011 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:25:54.011 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:25:54.011 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:25:54.011 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:25:54.011 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:25:54.011 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:25:54.235 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:25:54.504 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:25:54.499+0000 7f4239e6ba40 -1 osd.0 0 log_to_monitors true 2026-02-24T18:25:55.237 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:25:55.237 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:25:55.237 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:25:55.237 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:25:55.237 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:25:55.237 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:25:55.462 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1150958777,v1:127.0.0.1:6803/1150958777] [v2:127.0.0.1:6804/1150958777,v1:127.0.0.1:6805/1150958777] exists,up 7bbcb87b-5704-449d-b1bd-4831165eb2bb 2026-02-24T18:25:55.462 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:25:55.463 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:25:55.463 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:25:55.463 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:365: TEST_bluestore2: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-24T18:25:55.463 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:365: TEST_bluestore2: osd_pid0=135081 2026-02-24T18:25:55.463 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:367: TEST_bluestore2: sleep 5 2026-02-24T18:26:00.465 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:368: TEST_bluestore2: create_pool foo 16 2026-02-24T18:26:00.466 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 2026-02-24T18:26:00.733 INFO:tasks.workunit.client.0.trial172.stderr:pool 'foo' already exists 2026-02-24T18:26:00.745 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T18:26:01.746 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:370: TEST_bluestore2: retry = 0 2026-02-24T18:26:01.746 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 370: retry: command not found 2026-02-24T18:26:01.747 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:371: TEST_bluestore2: [[ '' -le 5 ]] 2026-02-24T18:26:01.747 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:373: TEST_bluestore2: timeout 60 rados bench -p foo 10 write --write-omap --no-cleanup 2026-02-24T18:26:01.769 INFO:tasks.workunit.client.0.trial172.stdout:hints = 1 2026-02-24T18:26:01.771 INFO:tasks.workunit.client.0.trial172.stdout:Maintaining 16 concurrent writes of 4194304 bytes to objects of size 4194304 for up to 10 seconds or 0 objects 2026-02-24T18:26:01.771 INFO:tasks.workunit.client.0.trial172.stdout:Object prefix: benchmark_data_trial172_137478 2026-02-24T18:26:01.814 INFO:tasks.workunit.client.0.trial172.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T18:26:01.815 INFO:tasks.workunit.client.0.trial172.stdout: 0 0 0 0 0 0 - 0 2026-02-24T18:26:02.815 INFO:tasks.workunit.client.0.trial172.stdout: 1 16 165 149 595.945 596 0.0944173 0.104489 2026-02-24T18:26:03.815 INFO:tasks.workunit.client.0.trial172.stdout: 2 16 357 341 681.941 768 0.0859978 0.09256 2026-02-24T18:26:04.815 INFO:tasks.workunit.client.0.trial172.stdout: 3 16 533 517 689.275 704 0.0838389 0.0913699 2026-02-24T18:26:05.815 INFO:tasks.workunit.client.0.trial172.stdout: 4 16 725 709 708.94 768 0.0944141 0.089234 2026-02-24T18:26:06.815 INFO:tasks.workunit.client.0.trial172.stdout: 5 16 927 911 728.741 808 0.0719447 0.0874814 2026-02-24T18:26:07.815 INFO:tasks.workunit.client.0.trial172.stdout: 6 16 1119 1103 735.272 768 0.0804784 0.0867812 2026-02-24T18:26:08.815 INFO:tasks.workunit.client.0.trial172.stdout: 7 15 1294 1279 730.793 704 0.106727 0.0871559 2026-02-24T18:26:09.816 INFO:tasks.workunit.client.0.trial172.stdout: 8 16 1487 1471 735.433 768 0.0932533 0.0866984 2026-02-24T18:26:10.816 INFO:tasks.workunit.client.0.trial172.stdout: 9 16 1685 1669 741.71 792 0.0780304 0.0858616 2026-02-24T18:26:11.816 INFO:tasks.workunit.client.0.trial172.stdout: 10 16 1877 1861 744.332 768 0.075814 0.0855718 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Total time run: 10.0489 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Total writes made: 1877 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Write size: 4194304 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Object size: 4194304 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Bandwidth (MB/sec): 747.144 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Stddev Bandwidth: 61.8083 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Max bandwidth (MB/sec): 808 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Min bandwidth (MB/sec): 596 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Average IOPS: 186 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Stddev IOPS: 15.4521 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Max IOPS: 202 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Min IOPS: 149 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Average Latency(s): 0.0855218 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Stddev Latency(s): 0.0121318 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Max latency(s): 0.125474 2026-02-24T18:26:12.816 INFO:tasks.workunit.client.0.trial172.stdout:Min latency(s): 0.0344877 2026-02-24T18:26:12.825 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:376: TEST_bluestore2: sleep 10 2026-02-24T18:26:22.829 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:378: TEST_bluestore2: jq .bluefs.db_used_bytes 2026-02-24T18:26:22.829 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:378: TEST_bluestore2: ceph tell osd.0 perf dump bluefs 2026-02-24T18:26:22.915 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:378: TEST_bluestore2: db_used=521142272 2026-02-24T18:26:22.915 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:379: TEST_bluestore2: ceph tell osd.0 perf dump bluefs 2026-02-24T18:26:22.915 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:379: TEST_bluestore2: jq .bluefs.slow_used_bytes 2026-02-24T18:26:22.998 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:379: TEST_bluestore2: spilled_over=184680448 2026-02-24T18:26:22.998 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:380: TEST_bluestore2: (( retry+=1 )) 2026-02-24T18:26:22.998 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:381: TEST_bluestore2: test 184680448 -eq 0 2026-02-24T18:26:22.999 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:381: TEST_bluestore2: break 2026-02-24T18:26:22.999 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:383: TEST_bluestore2: test 184680448 -gt 0 2026-02-24T18:26:22.999 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:385: TEST_bluestore2: kill 135081 2026-02-24T18:26:22.999 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:385: TEST_bluestore2: sleep 1 2026-02-24T18:26:23.999 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:385: TEST_bluestore2: kill 135081 2026-02-24T18:26:23.999 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 385: kill: (135081) - No such process 2026-02-24T18:26:24.000 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:386: TEST_bluestore2: ceph osd down 0 2026-02-24T18:26:24.210 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already down. 2026-02-24T18:26:24.220 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:388: TEST_bluestore2: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.db --dev-target td/osd-bluefs-volume-ops/0/block --command bluefs-bdev-migrate 2026-02-24T18:26:24.240 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:26:25.666 INFO:tasks.workunit.client.0.trial172.stdout: device removed:1 td/osd-bluefs-volume-ops/0/block.db 2026-02-24T18:26:25.682 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:393: TEST_bluestore2: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --command bluefs-bdev-sizes 2026-02-24T18:26:25.701 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:26:26.058 INFO:tasks.workunit.client.0.trial172.stdout:1 : device size 0x100000000(4 GiB) : using 0x29928000(665 MiB) : bluefs used 0x29910000(665 MiB) : non-bluefs used 0x18000(96 KiB) 2026-02-24T18:26:26.161 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:396: TEST_bluestore2: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --command fsck 2026-02-24T18:26:27.851 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:26:27.925 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:399: TEST_bluestore2: activate_osd td/osd-bluefs-volume-ops 0 2026-02-24T18:26:27.925 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:26:27.925 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:26:27.926 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-24T18:26:27.926 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:26:27.926 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-24T18:26:27.926 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=981d3548-b091-474d-b724-cb00f7e35350 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-24T18:26:27.926 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:26:27.926 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:26:27.926 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:26:27.926 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-24T18:26:27.926 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-24T18:26:27.926 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:26:27.926 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:26:27.926 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:26:27.926 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:26:27.926 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:26:27.927 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:26:27.927 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:26:27.927 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:26:27.927 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:26:27.927 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:26:27.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:26:27.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:26:27.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:26:27.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:26:27.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:26:27.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:26:27.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:26:27.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:26:27.928 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:26:27.929 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:26:27.929 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-24T18:26:27.929 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:26:27.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-24T18:26:27.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=981d3548-b091-474d-b724-cb00f7e35350 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:26:27.930 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-24T18:26:27.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-24T18:26:27.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:26:27.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:26:27.930 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:26:27.951 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:26:27.946+0000 7fc8cb5dea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:26:27.952 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:26:27.946+0000 7fc8cb5dea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:26:27.953 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:26:27.946+0000 7fc8cb5dea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:26:28.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-24T18:26:28.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:26:28.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:26:28.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:26:28.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:26:28.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:26:28.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:26:28.153 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:26:28.154 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:26:28.154 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:26:28.352 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:26:28.346+0000 7fc8cb5dea40 -1 Falling back to public interface 2026-02-24T18:26:28.367 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:26:29.368 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:26:29.369 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:26:29.369 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:26:29.369 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:26:29.369 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:26:29.369 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:26:29.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:26:30.593 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:26:30.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:26:30.594 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:26:30.595 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:26:30.595 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:26:30.595 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:26:30.814 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:26:31.052 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:26:31.046+0000 7fc8cb5dea40 -1 osd.0 9 log_to_monitors true 2026-02-24T18:26:31.815 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:26:31.815 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:26:31.815 INFO:tasks.workunit.client.0.trial172.stdout:3 2026-02-24T18:26:31.816 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-24T18:26:31.816 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:26:31.816 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:26:32.028 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 12 up_thru 12 down_at 9 last_clean_interval [5,9) [v2:127.0.0.1:6802/2946054059,v1:127.0.0.1:6803/2946054059] [v2:127.0.0.1:6804/2946054059,v1:127.0.0.1:6805/2946054059] exists,up 7bbcb87b-5704-449d-b1bd-4831165eb2bb 2026-02-24T18:26:32.028 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:26:32.028 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:26:32.028 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:26:32.028 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:400: TEST_bluestore2: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-24T18:26:32.029 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:400: TEST_bluestore2: osd_pid0=138709 2026-02-24T18:26:32.029 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:402: TEST_bluestore2: wait_for_clean 2026-02-24T18:26:32.029 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:26:32.030 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:26:32.030 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:26:32.030 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T18:26:32.031 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:26:32.031 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:26:32.031 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:26:32.031 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:26:32.031 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:26:32.133 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T18:26:32.133 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:26:32.133 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:26:32.133 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:26:32.133 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:26:32.133 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:26:32.345 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids=0 2026-02-24T18:26:32.346 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:26:32.346 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:26:32.346 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:26:32.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607555 2026-02-24T18:26:32.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607555 2026-02-24T18:26:32.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-51539607555' 2026-02-24T18:26:32.430 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:26:32.430 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-51539607555 2026-02-24T18:26:32.430 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:26:32.432 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:26:32.432 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-51539607555 2026-02-24T18:26:32.432 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:26:32.434 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607555 2026-02-24T18:26:32.434 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 51539607555 2026-02-24T18:26:32.434 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 51539607555' 2026-02-24T18:26:32.434 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:26:32.665 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 1 -lt 51539607555 2026-02-24T18:26:32.665 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T18:26:33.666 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T18:26:33.666 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:26:33.879 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607555 -lt 51539607555 2026-02-24T18:26:33.879 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:26:33.880 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:26:33.880 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:26:34.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-24T18:26:34.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:26:34.166 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:26:34.166 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:26:34.166 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:26:34.166 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:26:34.166 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:26:34.166 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:26:34.377 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-24T18:26:34.378 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:26:34.378 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:26:34.378 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:26:34.659 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-24T18:26:34.659 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T18:26:34.659 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T18:26:34.659 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:15: run: teardown td/osd-bluefs-volume-ops 2026-02-24T18:26:34.659 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:26:34.659 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:26:34.660 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-24T18:26:34.660 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:26:34.660 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:26:34.660 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:26:34.660 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:26:34.660 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:26:34.766 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:26:34.767 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:26:34.767 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:26:34.768 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:26:34.769 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:26:34.769 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:26:34.770 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:26:34.771 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:26:34.771 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:26:34.772 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:26:34.772 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:26:34.772 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:26:34.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:26:34.774 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-24T18:26:35.104 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:26:35.104 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:26:35.104 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:26:35.104 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.88188 2026-02-24T18:26:35.105 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:26:35.106 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:26:35.106 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:12: run: for func in $funcs 2026-02-24T18:26:35.106 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:13: run: setup td/osd-bluefs-volume-ops 2026-02-24T18:26:35.106 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:26:35.106 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-bluefs-volume-ops 2026-02-24T18:26:35.106 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:26:35.106 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T18:26:35.106 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-24T18:26:35.107 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:26:35.107 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:26:35.107 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:26:35.107 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:26:35.107 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:26:35.109 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:26:35.109 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:26:35.110 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:26:35.110 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:26:35.111 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:26:35.111 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:26:35.112 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:26:35.113 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:26:35.113 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:26:35.113 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:26:35.114 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:26:35.115 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:26:35.116 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T18:26:35.116 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-24T18:26:35.117 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:26:35.118 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:26:35.118 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:26:35.118 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.88188 2026-02-24T18:26:35.119 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:26:35.119 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:26:35.119 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-bluefs-volume-ops 2026-02-24T18:26:35.120 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T18:26:35.121 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:26:35.121 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:26:35.121 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.88188 2026-02-24T18:26:35.122 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T18:26:35.123 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T18:26:35.123 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T18:26:35.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-bluefs-volume-ops 1' TERM HUP INT 2026-02-24T18:26:35.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: TEST_bluestore_expand td/osd-bluefs-volume-ops 2026-02-24T18:26:35.124 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:406: TEST_bluestore_expand: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:26:35.124 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:408: TEST_bluestore_expand: ulimit -n 2026-02-24T18:26:35.125 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:408: TEST_bluestore_expand: local flimit=4096 2026-02-24T18:26:35.125 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:409: TEST_bluestore_expand: '[' 4096 -lt 1536 ']' 2026-02-24T18:26:35.125 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:412: TEST_bluestore_expand: export CEPH_MON=127.0.0.1:7146 2026-02-24T18:26:35.125 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:412: TEST_bluestore_expand: CEPH_MON=127.0.0.1:7146 2026-02-24T18:26:35.125 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:413: TEST_bluestore_expand: export CEPH_ARGS 2026-02-24T18:26:35.125 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:414: TEST_bluestore_expand: uuidgen 2026-02-24T18:26:35.127 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:414: TEST_bluestore_expand: CEPH_ARGS+='--fsid=e4c22fc4-1653-4e1f-849b-e97e11ae8d8e --auth-supported=none ' 2026-02-24T18:26:35.127 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:415: TEST_bluestore_expand: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-24T18:26:35.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:416: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_size=4294967296 ' 2026-02-24T18:26:35.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:417: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_db_create=true ' 2026-02-24T18:26:35.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:418: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_db_size=1073741824 ' 2026-02-24T18:26:35.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:419: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_wal_create=false ' 2026-02-24T18:26:35.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:420: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_fsck_on_mount=true ' 2026-02-24T18:26:35.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:421: TEST_bluestore_expand: CEPH_ARGS+='--osd_pool_default_size=1 ' 2026-02-24T18:26:35.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:422: TEST_bluestore_expand: CEPH_ARGS+='--osd_pool_default_min_size=1 ' 2026-02-24T18:26:35.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:423: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_debug_enforce_settings=ssd ' 2026-02-24T18:26:35.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:425: TEST_bluestore_expand: run_mon td/osd-bluefs-volume-ops a 2026-02-24T18:26:35.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:26:35.128 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T18:26:35.129 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T18:26:35.129 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T18:26:35.129 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-bluefs-volume-ops/a 2026-02-24T18:26:35.129 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bluefs-volume-ops/a --run-dir=td/osd-bluefs-volume-ops 2026-02-24T18:26:35.151 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T18:26:35.151 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:26:35.151 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:26:35.152 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:26:35.152 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:26:35.152 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:26:35.152 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:26:35.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bluefs-volume-ops/a '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bluefs-volume-ops/log --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T18:26:35.189 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T18:26:35.190 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T18:26:35.190 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:26:35.190 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:26:35.190 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T18:26:35.190 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T18:26:35.191 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:26:35.191 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:26:35.191 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:26:35.191 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:26:35.191 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:26:35.191 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:26:35.191 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.88188/ceph-mon.a.asok 2026-02-24T18:26:35.192 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:26:35.192 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.88188/ceph-mon.a.asok config get fsid 2026-02-24T18:26:35.250 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T18:26:35.250 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T18:26:35.250 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T18:26:35.251 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T18:26:35.251 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T18:26:35.251 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T18:26:35.251 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T18:26:35.251 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T18:26:35.251 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T18:26:35.251 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:26:35.251 INFO:tasks.workunit.client.0.trial172.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:26:35.251 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.88188/ceph-mon.a.asok 2026-02-24T18:26:35.252 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T18:26:35.252 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.88188/ceph-mon.a.asok config get mon_host 2026-02-24T18:26:35.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:426: TEST_bluestore_expand: run_mgr td/osd-bluefs-volume-ops x 2026-02-24T18:26:35.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:26:35.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T18:26:35.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T18:26:35.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T18:26:35.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-bluefs-volume-ops/x 2026-02-24T18:26:35.306 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T18:26:35.424 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T18:26:35.424 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:26:35.424 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:26:35.425 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:26:35.425 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:26:35.425 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:26:35.425 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:26:35.426 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:26:35.427 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bluefs-volume-ops/x '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T18:26:35.448 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:427: TEST_bluestore_expand: run_osd td/osd-bluefs-volume-ops 0 2026-02-24T18:26:35.448 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:26:35.448 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T18:26:35.448 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T18:26:35.448 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T18:26:35.448 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-24T18:26:35.448 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=981d3548-b091-474d-b724-cb00f7e35350 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=e4c22fc4-1653-4e1f-849b-e97e11ae8d8e --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-24T18:26:35.448 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:26:35.448 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:26:35.448 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:26:35.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-24T18:26:35.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-24T18:26:35.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T18:26:35.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T18:26:35.449 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:26:35.449 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T18:26:35.449 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:26:35.449 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:26:35.449 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:26:35.449 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:26:35.449 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:26:35.449 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:26:35.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:26:35.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:26:35.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T18:26:35.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T18:26:35.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:26:35.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:26:35.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:26:35.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:26:35.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:26:35.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:26:35.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T18:26:35.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T18:26:35.450 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-24T18:26:35.451 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T18:26:35.451 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c285a7ca-a7fc-4833-a47f-ac838c9f7845 2026-02-24T18:26:35.451 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c285a7ca-a7fc-4833-a47f-ac838c9f7845' 2026-02-24T18:26:35.452 INFO:tasks.workunit.client.0.trial172.stdout:add osd0 c285a7ca-a7fc-4833-a47f-ac838c9f7845 2026-02-24T18:26:35.452 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T18:26:35.466 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBb7Z1poQi5GxAAa9obux/9fltKJ2+Xy1TEwQ== 2026-02-24T18:26:35.467 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBb7Z1poQi5GxAAa9obux/9fltKJ2+Xy1TEwQ=="}' 2026-02-24T18:26:35.467 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c285a7ca-a7fc-4833-a47f-ac838c9f7845 -i td/osd-bluefs-volume-ops/0/new.json 2026-02-24T18:26:35.585 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:26:35.592 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-bluefs-volume-ops/0/new.json 2026-02-24T18:26:35.594 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=981d3548-b091-474d-b724-cb00f7e35350 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=e4c22fc4-1653-4e1f-849b-e97e11ae8d8e --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBb7Z1poQi5GxAAa9obux/9fltKJ2+Xy1TEwQ== --osd-uuid c285a7ca-a7fc-4833-a47f-ac838c9f7845 2026-02-24T18:26:35.614 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:26:35.606+0000 7f16a1fa8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:26:35.615 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:26:35.610+0000 7f16a1fa8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:26:35.616 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:26:35.610+0000 7f16a1fa8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:26:35.616 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:26:35.610+0000 7f16a1fa8a40 -1 bdev(0x559edf664800 td/osd-bluefs-volume-ops/0/block) open stat got: (1) Operation not permitted 2026-02-24T18:26:35.616 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:26:35.610+0000 7f16a1fa8a40 -1 bluestore(td/osd-bluefs-volume-ops/0) _read_fsid unparsable uuid 2026-02-24T18:26:37.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-bluefs-volume-ops/0/keyring 2026-02-24T18:26:37.018 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T18:26:37.019 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T18:26:37.019 INFO:tasks.workunit.client.0.trial172.stdout:adding osd0 key to auth repository 2026-02-24T18:26:37.020 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-bluefs-volume-ops/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T18:26:37.137 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T18:26:37.137 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:26:37.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=981d3548-b091-474d-b724-cb00f7e35350 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=e4c22fc4-1653-4e1f-849b-e97e11ae8d8e --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:26:37.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T18:26:37.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T18:26:37.138 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T18:26:37.157 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:26:37.150+0000 7f2deb801a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:26:37.158 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:26:37.154+0000 7f2deb801a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:26:37.159 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:26:37.154+0000 7f2deb801a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:26:37.259 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T18:26:37.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:26:37.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:26:37.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:26:37.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:26:37.260 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:26:37.260 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:26:37.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:26:37.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:26:37.261 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:26:37.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:26:37.703 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:26:37.698+0000 7f2deb801a40 -1 Falling back to public interface 2026-02-24T18:26:38.377 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:26:38.378 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:26:38.378 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:26:38.379 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:26:38.379 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:26:38.379 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:26:38.605 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:26:38.795 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:26:38.790+0000 7f2deb801a40 -1 osd.0 0 log_to_monitors true 2026-02-24T18:26:39.606 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:26:39.606 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:26:39.606 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:26:39.607 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:26:39.607 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:26:39.607 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:26:39.833 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2520996828,v1:127.0.0.1:6801/2520996828] [v2:127.0.0.1:6802/2520996828,v1:127.0.0.1:6803/2520996828] exists,up c285a7ca-a7fc-4833-a47f-ac838c9f7845 2026-02-24T18:26:39.834 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:26:39.834 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:26:39.834 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:26:39.834 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:428: TEST_bluestore_expand: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-24T18:26:39.835 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:428: TEST_bluestore_expand: osd_pid0=141269 2026-02-24T18:26:39.835 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:430: TEST_bluestore_expand: sleep 5 2026-02-24T18:26:44.837 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:431: TEST_bluestore_expand: create_pool foo 16 2026-02-24T18:26:44.837 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 2026-02-24T18:26:45.105 INFO:tasks.workunit.client.0.trial172.stderr:pool 'foo' already exists 2026-02-24T18:26:45.114 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T18:26:46.116 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:434: TEST_bluestore_expand: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-24T18:26:46.140 INFO:tasks.workunit.client.0.trial172.stdout:hints = 1 2026-02-24T18:26:46.140 INFO:tasks.workunit.client.0.trial172.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-24T18:26:46.140 INFO:tasks.workunit.client.0.trial172.stdout:Object prefix: benchmark_data_trial172_143396 2026-02-24T18:26:46.140 INFO:tasks.workunit.client.0.trial172.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T18:26:46.140 INFO:tasks.workunit.client.0.trial172.stdout: 0 0 0 0 0 0 - 0 2026-02-24T18:26:47.140 INFO:tasks.workunit.client.0.trial172.stdout: 1 16 15996 15980 62.4168 62.4219 0.000949753 0.00100006 2026-02-24T18:26:48.140 INFO:tasks.workunit.client.0.trial172.stdout: 2 16 32128 32112 62.7137 63.0156 0.00105353 0.000995317 2026-02-24T18:26:49.140 INFO:tasks.workunit.client.0.trial172.stdout: 3 16 47806 47790 62.2215 61.2422 0.000644627 0.00100336 2026-02-24T18:26:50.140 INFO:tasks.workunit.client.0.trial172.stdout: 4 16 63408 63392 61.9011 60.9453 0.000833427 0.00100851 2026-02-24T18:26:51.140 INFO:tasks.workunit.client.0.trial172.stdout: 5 16 79117 79101 61.7926 61.3633 0.000507121 0.00101036 2026-02-24T18:26:52.141 INFO:tasks.workunit.client.0.trial172.stdout: 6 16 94794 94778 61.6994 61.2383 0.000949452 0.00101191 2026-02-24T18:26:53.141 INFO:tasks.workunit.client.0.trial172.stdout: 7 15 109237 109222 60.9448 56.4219 0.00192735 0.00102446 2026-02-24T18:26:54.140 INFO:tasks.workunit.client.0.trial172.stdout: 8 15 123131 123116 60.1104 54.2734 0.00141529 0.00103872 2026-02-24T18:26:55.140 INFO:tasks.workunit.client.0.trial172.stdout: 9 16 137097 137081 59.4922 54.5508 0.000946938 0.00104951 2026-02-24T18:26:56.141 INFO:tasks.workunit.client.0.trial172.stdout: 10 15 151432 151417 59.1426 56 0.0010468 0.00105576 2026-02-24T18:26:57.141 INFO:tasks.workunit.client.0.trial172.stdout: 11 16 165481 165465 58.7542 54.875 0.00136461 0.00106275 2026-02-24T18:26:58.141 INFO:tasks.workunit.client.0.trial172.stdout: 12 16 179034 179018 58.2694 52.9414 0.00110393 0.0010716 2026-02-24T18:26:59.141 INFO:tasks.workunit.client.0.trial172.stdout: 13 16 192697 192681 57.8923 53.3711 0.00107452 0.00107861 2026-02-24T18:27:00.141 INFO:tasks.workunit.client.0.trial172.stdout: 14 16 206259 206243 57.5409 52.9766 0.00142519 0.0010852 2026-02-24T18:27:01.141 INFO:tasks.workunit.client.0.trial172.stdout: 15 16 219956 219940 57.2715 53.5039 0.00078173 0.00109033 2026-02-24T18:27:02.141 INFO:tasks.workunit.client.0.trial172.stdout: 16 15 233575 233560 57.017 53.2031 0.00123153 0.00109521 2026-02-24T18:27:03.141 INFO:tasks.workunit.client.0.trial172.stdout: 17 16 247142 247126 56.78 52.9922 0.0013191 0.00109976 2026-02-24T18:27:04.141 INFO:tasks.workunit.client.0.trial172.stdout: 18 15 260725 260710 56.5732 53.0625 0.00093167 0.00110378 2026-02-24T18:27:05.141 INFO:tasks.workunit.client.0.trial172.stdout: 19 16 274408 274392 56.4084 53.4453 0.00106096 0.001107 2026-02-24T18:27:06.142 INFO:tasks.workunit.client.0.trial172.stdout:2026-02-24T18:27:06.140663+0000 min lat: 0.000355199 max lat: 0.00516914 avg lat: 0.00111005 2026-02-24T18:27:06.142 INFO:tasks.workunit.client.0.trial172.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-24T18:27:06.142 INFO:tasks.workunit.client.0.trial172.stdout: 20 16 288058 288042 56.2538 53.3203 0.00267373 0.00111005 2026-02-24T18:27:07.142 INFO:tasks.workunit.client.0.trial172.stdout: 21 15 301240 301225 56.027 51.4961 0.00102931 0.00111455 2026-02-24T18:27:08.142 INFO:tasks.workunit.client.0.trial172.stdout: 22 16 314529 314513 55.8395 51.9062 0.000866078 0.00111829 2026-02-24T18:27:09.142 INFO:tasks.workunit.client.0.trial172.stdout: 23 16 316698 316682 53.7801 8.47266 0.0120461 0.00115902 2026-02-24T18:27:10.142 INFO:tasks.workunit.client.0.trial172.stdout: 24 16 317247 317231 51.6286 2.14453 0.0228662 0.00120495 2026-02-24T18:27:11.142 INFO:tasks.workunit.client.0.trial172.stdout: 25 16 318085 318069 49.6944 3.27344 0.0241868 0.00125607 2026-02-24T18:27:12.142 INFO:tasks.workunit.client.0.trial172.stdout: 26 16 318341 318325 47.8216 1 0.119667 0.00130109 2026-02-24T18:27:13.142 INFO:tasks.workunit.client.0.trial172.stdout: 27 16 318476 318460 46.0699 0.527344 0.108235 0.00135246 2026-02-24T18:27:14.142 INFO:tasks.workunit.client.0.trial172.stdout: 28 16 326992 326976 45.6126 33.2656 0.000999105 0.00136926 2026-02-24T18:27:15.142 INFO:tasks.workunit.client.0.trial172.stdout: 29 16 340092 340076 45.8041 51.1719 0.0013225 0.00136353 2026-02-24T18:27:16.142 INFO:tasks.workunit.client.0.trial172.stdout:Total time run: 30.0006 2026-02-24T18:27:16.143 INFO:tasks.workunit.client.0.trial172.stdout:Total writes made: 353731 2026-02-24T18:27:16.143 INFO:tasks.workunit.client.0.trial172.stdout:Write size: 4096 2026-02-24T18:27:16.143 INFO:tasks.workunit.client.0.trial172.stdout:Object size: 4096 2026-02-24T18:27:16.143 INFO:tasks.workunit.client.0.trial172.stdout:Bandwidth (MB/sec): 46.0578 2026-02-24T18:27:16.143 INFO:tasks.workunit.client.0.trial172.stdout:Stddev Bandwidth: 20.6 2026-02-24T18:27:16.143 INFO:tasks.workunit.client.0.trial172.stdout:Max bandwidth (MB/sec): 63.0156 2026-02-24T18:27:16.143 INFO:tasks.workunit.client.0.trial172.stdout:Min bandwidth (MB/sec): 0.527344 2026-02-24T18:27:16.144 INFO:tasks.workunit.client.0.trial172.stdout:Average IOPS: 11790 2026-02-24T18:27:16.144 INFO:tasks.workunit.client.0.trial172.stdout:Stddev IOPS: 5273.61 2026-02-24T18:27:16.144 INFO:tasks.workunit.client.0.trial172.stdout:Max IOPS: 16132 2026-02-24T18:27:16.144 INFO:tasks.workunit.client.0.trial172.stdout:Min IOPS: 135 2026-02-24T18:27:16.144 INFO:tasks.workunit.client.0.trial172.stdout:Average Latency(s): 0.00135601 2026-02-24T18:27:16.144 INFO:tasks.workunit.client.0.trial172.stdout:Stddev Latency(s): 0.00416892 2026-02-24T18:27:16.144 INFO:tasks.workunit.client.0.trial172.stdout:Max latency(s): 0.182715 2026-02-24T18:27:16.144 INFO:tasks.workunit.client.0.trial172.stdout:Min latency(s): 0.000355199 2026-02-24T18:27:16.146 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:435: TEST_bluestore_expand: sleep 5 2026-02-24T18:27:21.149 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:437: TEST_bluestore_expand: ceph tell osd.0 perf dump bluefs 2026-02-24T18:27:21.149 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:437: TEST_bluestore_expand: jq .bluefs.slow_total_bytes 2026-02-24T18:27:21.237 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:437: TEST_bluestore_expand: total_space_before=4294967296 2026-02-24T18:27:21.238 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: ceph tell osd.0 bluestore bluefs device info 2026-02-24T18:27:21.238 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: grep BDEV_SLOW -A 2 2026-02-24T18:27:21.238 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: grep free 2026-02-24T18:27:21.238 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: cut -d: -f 2 2026-02-24T18:27:21.238 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: cut -d, -f 1 2026-02-24T18:27:21.238 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: cut '-d ' -f 2 2026-02-24T18:27:21.322 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: free_space_before=2845986816 2026-02-24T18:27:21.322 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: kill 141269 2026-02-24T18:27:21.322 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: sleep 1 2026-02-24T18:27:22.322 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: kill 141269 2026-02-24T18:27:22.322 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 441: kill: (141269) - No such process 2026-02-24T18:27:22.322 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:442: TEST_bluestore_expand: ceph osd down 0 2026-02-24T18:27:22.540 INFO:tasks.workunit.client.0.trial172.stderr:osd.0 is already down. 2026-02-24T18:27:22.550 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:445: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 allocmap 2026-02-24T18:27:22.574 INFO:tasks.workunit.client.0.trial172.stdout:allocmap bluestore.allocmap 2026-02-24T18:28:07.175 INFO:tasks.workunit.client.0.trial172.stdout:allocmap success 2026-02-24T18:28:07.204 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:448: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 fsck 2026-02-24T18:28:53.799 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:28:53.829 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:450: TEST_bluestore_expand: requested_space=4294967296 2026-02-24T18:28:53.830 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:451: TEST_bluestore_expand: truncate td/osd-bluefs-volume-ops/0/block -s 4294967296 2026-02-24T18:28:53.830 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:452: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 bluefs-bdev-expand 2026-02-24T18:28:53.848 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:29:37.168 INFO:tasks.workunit.client.0.trial172.stdout:1 : device size 0x40000000(1 GiB) : using 0x29a00000(666 MiB) 2026-02-24T18:29:37.168 INFO:tasks.workunit.client.0.trial172.stdout:2 : device size 0x100000000(4 GiB) : using 0x565dd000(1.3 GiB) : bluefs used 0x0(0 B) : non-bluefs used 0x565dd000(1.3 GiB) 2026-02-24T18:29:37.168 INFO:tasks.workunit.client.0.trial172.stdout:Expanding DB/WAL... 2026-02-24T18:29:37.169 INFO:tasks.workunit.client.0.trial172.stdout:1 : nothing to do, skipped 2026-02-24T18:29:37.169 INFO:tasks.workunit.client.0.trial172.stdout:2 : nothing to do, skipped 2026-02-24T18:29:37.311 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:455: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 fsck 2026-02-24T18:30:23.635 INFO:tasks.workunit.client.0.trial172.stdout:fsck success 2026-02-24T18:30:23.665 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:458: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 qfsck 2026-02-24T18:30:23.682 INFO:tasks.workunit.client.0.trial172.stdout:qfsck bluestore.quick-fsck 2026-02-24T18:31:08.145 INFO:tasks.workunit.client.0.trial172.stdout:qfsck success 2026-02-24T18:31:08.163 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:460: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 bluefs-bdev-sizes 2026-02-24T18:31:08.181 INFO:tasks.workunit.client.0.trial172.stdout:inferring bluefs devices from bluestore path 2026-02-24T18:31:51.979 INFO:tasks.workunit.client.0.trial172.stdout:1 : device size 0x40000000(1 GiB) : using 0x29a00000(666 MiB) 2026-02-24T18:31:51.980 INFO:tasks.workunit.client.0.trial172.stdout:2 : device size 0x100000000(4 GiB) : using 0x565dd000(1.3 GiB) : bluefs used 0x0(0 B) : non-bluefs used 0x565dd000(1.3 GiB) 2026-02-24T18:31:52.150 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:462: TEST_bluestore_expand: activate_osd td/osd-bluefs-volume-ops 0 2026-02-24T18:31:52.150 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:31:52.150 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T18:31:52.150 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-24T18:31:52.150 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T18:31:52.150 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-24T18:31:52.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=981d3548-b091-474d-b724-cb00f7e35350 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=e4c22fc4-1653-4e1f-849b-e97e11ae8d8e --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-24T18:31:52.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T18:31:52.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T18:31:52.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T18:31:52.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-24T18:31:52.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-24T18:31:52.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T18:31:52.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T18:31:52.151 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-24T18:31:52.151 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T18:31:52.151 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T18:31:52.152 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T18:31:52.152 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T18:31:52.152 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:31:52.152 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:31:52.152 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:31:52.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' 2026-02-24T18:31:52.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T18:31:52.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-24T18:31:52.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-24T18:31:52.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T18:31:52.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T18:31:52.152 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T18:31:52.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T18:31:52.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T18:31:52.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T18:31:52.153 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-24T18:31:52.153 INFO:tasks.workunit.client.0.trial172.stdout:start osd.0 2026-02-24T18:31:52.154 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-24T18:31:52.154 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=a389b28e-04ad-4ead-8fc5-d4290f34b76d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=0 --fsid=981d3548-b091-474d-b724-cb00f7e35350 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=e4c22fc4-1653-4e1f-849b-e97e11ae8d8e --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.88188/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T18:31:52.154 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-24T18:31:52.155 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-24T18:31:52.155 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T18:31:52.155 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T18:31:52.155 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T18:31:52.173 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:31:52.166+0000 7f3555f3ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:31:52.174 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:31:52.166+0000 7f3555f3ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:31:52.175 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:31:52.170+0000 7f3555f3ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T18:31:52.374 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-24T18:31:52.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T18:31:52.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T18:31:52.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T18:31:52.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T18:31:52.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:31:52.375 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T18:31:52.375 INFO:tasks.workunit.client.0.trial172.stdout:0 2026-02-24T18:31:52.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:31:52.376 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:31:52.588 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:31:52.582+0000 7f3555f3ca40 -1 Falling back to public interface 2026-02-24T18:31:52.589 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:31:53.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:31:53.591 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:31:53.592 INFO:tasks.workunit.client.0.trial172.stdout:1 2026-02-24T18:31:53.593 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T18:31:53.593 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:31:53.593 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:31:53.818 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:31:54.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:31:54.820 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:31:54.820 INFO:tasks.workunit.client.0.trial172.stdout:2 2026-02-24T18:31:54.821 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T18:31:54.821 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:31:54.821 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:31:55.044 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:31:56.045 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:31:56.046 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:31:56.046 INFO:tasks.workunit.client.0.trial172.stdout:3 2026-02-24T18:31:56.047 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-24T18:31:56.047 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:31:56.047 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:31:56.268 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:31:57.269 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:31:57.269 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:31:57.269 INFO:tasks.workunit.client.0.trial172.stdout:4 2026-02-24T18:31:57.269 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-24T18:31:57.269 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:31:57.269 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:31:57.490 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:31:58.492 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:31:58.492 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:31:58.492 INFO:tasks.workunit.client.0.trial172.stdout:5 2026-02-24T18:31:58.493 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-24T18:31:58.493 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:31:58.493 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:31:58.716 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:31:59.717 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:31:59.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:31:59.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-24T18:31:59.718 INFO:tasks.workunit.client.0.trial172.stdout:6 2026-02-24T18:31:59.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:31:59.718 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:31:59.939 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:32:00.941 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:32:00.941 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:32:00.941 INFO:tasks.workunit.client.0.trial172.stdout:7 2026-02-24T18:32:00.941 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-24T18:32:00.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:32:00.942 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:32:01.162 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:32:02.164 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:32:02.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:32:02.165 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2026-02-24T18:32:02.165 INFO:tasks.workunit.client.0.trial172.stdout:8 2026-02-24T18:32:02.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:32:02.166 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:32:02.383 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:32:03.384 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:32:03.384 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:32:03.384 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2026-02-24T18:32:03.384 INFO:tasks.workunit.client.0.trial172.stdout:9 2026-02-24T18:32:03.385 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:32:03.385 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:32:03.606 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:32:04.607 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:32:04.608 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:32:04.608 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2026-02-24T18:32:04.608 INFO:tasks.workunit.client.0.trial172.stdout:10 2026-02-24T18:32:04.609 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:32:04.609 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:32:04.824 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:32:05.826 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:32:05.826 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:32:05.827 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 11 2026-02-24T18:32:05.827 INFO:tasks.workunit.client.0.trial172.stdout:11 2026-02-24T18:32:05.828 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:32:05.828 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:32:06.046 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T18:32:06.722 INFO:tasks.workunit.client.0.trial172.stderr:2026-02-24T18:32:06.714+0000 7f3555f3ca40 -1 osd.0 10 log_to_monitors true 2026-02-24T18:32:07.048 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T18:32:07.048 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T18:32:07.049 INFO:tasks.workunit.client.0.trial172.stdout:12 2026-02-24T18:32:07.049 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 12 2026-02-24T18:32:07.049 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T18:32:07.050 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T18:32:07.263 INFO:tasks.workunit.client.0.trial172.stdout:osd.0 up in weight 1 up_from 13 up_thru 13 down_at 10 last_clean_interval [5,10) [v2:127.0.0.1:6800/3992570019,v1:127.0.0.1:6801/3992570019] [v2:127.0.0.1:6802/3992570019,v1:127.0.0.1:6803/3992570019] exists,up c285a7ca-a7fc-4833-a47f-ac838c9f7845 2026-02-24T18:32:07.263 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T18:32:07.264 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T18:32:07.264 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T18:32:07.264 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:463: TEST_bluestore_expand: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-24T18:32:07.265 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:463: TEST_bluestore_expand: osd_pid0=145772 2026-02-24T18:32:07.265 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:465: TEST_bluestore_expand: wait_for_clean 2026-02-24T18:32:07.265 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T18:32:07.265 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T18:32:07.265 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T18:32:07.265 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T18:32:07.266 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T18:32:07.266 INFO:tasks.workunit.client.0.trial172.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T18:32:07.267 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T18:32:07.267 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T18:32:07.267 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T18:32:07.359 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T18:32:07.360 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T18:32:07.360 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T18:32:07.360 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T18:32:07.360 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T18:32:07.360 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T18:32:07.581 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids=0 2026-02-24T18:32:07.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T18:32:07.582 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T18:32:07.582 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T18:32:07.668 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574851 2026-02-24T18:32:07.669 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574851 2026-02-24T18:32:07.669 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-55834574851' 2026-02-24T18:32:07.669 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T18:32:07.669 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-55834574851 2026-02-24T18:32:07.669 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T18:32:07.670 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T18:32:07.671 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-55834574851 2026-02-24T18:32:07.671 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T18:32:07.672 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574851 2026-02-24T18:32:07.672 INFO:tasks.workunit.client.0.trial172.stdout:waiting osd.0 seq 55834574851 2026-02-24T18:32:07.672 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 55834574851' 2026-02-24T18:32:07.673 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T18:32:07.900 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-24T18:32:07.901 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T18:32:07.901 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:32:07.901 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:32:08.197 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-24T18:32:08.197 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T18:32:08.197 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T18:32:08.197 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T18:32:08.197 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T18:32:08.197 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T18:32:08.198 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T18:32:08.198 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T18:32:08.414 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-24T18:32:08.415 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T18:32:08.415 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T18:32:08.415 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T18:32:08.710 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-24T18:32:08.710 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T18:32:08.711 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T18:32:08.711 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:467: TEST_bluestore_expand: ceph tell osd.0 perf dump bluefs 2026-02-24T18:32:08.711 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:467: TEST_bluestore_expand: jq .bluefs.slow_total_bytes 2026-02-24T18:32:08.797 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:467: TEST_bluestore_expand: total_space_after=4294967296 2026-02-24T18:32:08.798 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: ceph tell osd.0 bluestore bluefs device info 2026-02-24T18:32:08.798 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: grep BDEV_SLOW -A 2 2026-02-24T18:32:08.798 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: grep free 2026-02-24T18:32:08.798 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: cut -d: -f 2 2026-02-24T18:32:08.798 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: cut -d, -f 1 2026-02-24T18:32:08.798 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: cut '-d ' -f 2 2026-02-24T18:32:08.885 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: free_space_after=2819272704 2026-02-24T18:32:08.885 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:470: TEST_bluestore_expand: '[' 4294967296 '!=' 4294967296 ']' 2026-02-24T18:32:08.885 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:476: TEST_bluestore_expand: total_space_added=0 2026-02-24T18:32:08.885 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:477: TEST_bluestore_expand: free_space_added=-26714112 2026-02-24T18:32:08.885 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:479: TEST_bluestore_expand: let 'new_used_space=(0 - -26714112)' 2026-02-24T18:32:08.885 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:480: TEST_bluestore_expand: echo 26714112 2026-02-24T18:32:08.886 INFO:tasks.workunit.client.0.trial172.stdout:26714112 2026-02-24T18:32:08.886 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:482: TEST_bluestore_expand: '[' 26714112 -gt 131072 ']' 2026-02-24T18:32:08.886 INFO:tasks.workunit.client.0.trial172.stdout:total_space_added = 0 2026-02-24T18:32:08.887 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:483: TEST_bluestore_expand: echo 'total_space_added = 0' 2026-02-24T18:32:08.887 INFO:tasks.workunit.client.0.trial172.stdout:free_space_added = -26714112 2026-02-24T18:32:08.888 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:484: TEST_bluestore_expand: echo 'free_space_added = -26714112' 2026-02-24T18:32:08.889 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:485: TEST_bluestore_expand: return 1 2026-02-24T18:32:08.889 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: return 1 2026-02-24T18:32:08.889 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2431: main: code=1 2026-02-24T18:32:08.889 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2433: main: teardown td/osd-bluefs-volume-ops 1 2026-02-24T18:32:08.889 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-24T18:32:08.889 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=1 2026-02-24T18:32:08.889 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-24T18:32:08.889 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T18:32:08.889 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T18:32:08.889 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T18:32:08.889 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T18:32:08.890 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T18:32:09.194 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T18:32:09.195 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T18:32:09.196 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T18:32:09.196 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T18:32:09.198 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T18:32:09.198 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T18:32:09.199 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T18:32:09.200 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:32:09.200 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T18:32:09.200 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T18:32:09.201 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:32:09.202 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:32:09.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 1 = 1 ']' 2026-02-24T18:32:09.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:181: teardown: '[' -n '' ']' 2026-02-24T18:32:09.203 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:185: teardown: mkdir -p /home/ubuntu/cephtest/archive/log 2026-02-24T18:32:09.204 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:186: teardown: mv td/osd-bluefs-volume-ops/bluestore_tool.log td/osd-bluefs-volume-ops/mgr.x.log td/osd-bluefs-volume-ops/mon.a.log td/osd-bluefs-volume-ops/osd.0.log /home/ubuntu/cephtest/archive/log 2026-02-24T18:32:09.206 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-24T18:32:09.274 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T18:32:09.274 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T18:32:09.274 INFO:tasks.workunit.client.0.trial172.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.88188 2026-02-24T18:32:09.274 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.88188 2026-02-24T18:32:09.275 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T18:32:09.275 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T18:32:09.275 INFO:tasks.workunit.client.0.trial172.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2434: main: return 1 2026-02-24T18:32:09.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:32:09.277 INFO:tasks.workunit:Stopping ['osd'] on client.0... 2026-02-24T18:32:09.277 DEBUG:teuthology.orchestra.run.trial172:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-24T18:32:09.635 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_618c318a7a0b287ab5aeff207b7bf1d73ef45044/qa/tasks/workunit.py", line 144, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_618c318a7a0b287ab5aeff207b7bf1d73ef45044/qa/tasks/workunit.py", line 292, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_618c318a7a0b287ab5aeff207b7bf1d73ef45044/qa/tasks/workunit.py", line 433, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test osd/osd-bluefs-volume-ops.sh) on trial172 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=618c318a7a0b287ab5aeff207b7bf1d73ef45044 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh' 2026-02-24T18:32:09.642 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-24T18:32:09.658 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-24T18:32:09.658 DEBUG:teuthology.orchestra.run.trial172:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-24T18:32:09.767 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-24T18:32:09.777 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-24T18:32:09.778 DEBUG:teuthology.orchestra.run.trial172:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T18:32:10.143 INFO:teuthology.orchestra.run.trial172.stdout: remote refid st t when poll reach delay offset jitter 2026-02-24T18:32:10.143 INFO:teuthology.orchestra.run.trial172.stdout:============================================================================== 2026-02-24T18:32:10.143 INFO:teuthology.orchestra.run.trial172.stdout:+204.2.134.173 44.24.199.34 3 u 62 64 377 68.223 +0.990 0.310 2026-02-24T18:32:10.144 INFO:teuthology.orchestra.run.trial172.stdout:+mail.mlevin.net 132.163.96.2 2 u 57 64 377 40.970 -1.158 0.319 2026-02-24T18:32:10.144 INFO:teuthology.orchestra.run.trial172.stdout:*ntp1.wiktel.com .PPS. 1 u 59 64 377 38.748 +1.609 0.673 2026-02-24T18:32:10.144 INFO:teuthology.orchestra.run.trial172.stdout:-172-234-37-140. 82.35.162.146 2 u 65 64 377 7.568 -13.389 4.132 2026-02-24T18:32:10.145 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-24T18:32:10.166 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-24T18:32:10.166 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-24T18:32:10.181 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-24T18:32:10.195 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-24T18:32:10.209 INFO:teuthology.task.internal:Duration was 1653.446488 seconds 2026-02-24T18:32:10.209 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-24T18:32:10.223 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-24T18:32:10.224 DEBUG:teuthology.orchestra.run.trial172:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-24T18:32:10.250 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-24T18:32:10.250 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial172.front.sepia.ceph.com 2026-02-24T18:32:10.250 DEBUG:teuthology.orchestra.run.trial172:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-24T18:32:10.296 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-24T18:32:10.296 DEBUG:teuthology.orchestra.run.trial172:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T18:32:10.563 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-24T18:32:10.564 DEBUG:teuthology.orchestra.run.trial172:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T18:32:10.574 INFO:teuthology.orchestra.run.trial172.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:32:10.575 INFO:teuthology.orchestra.run.trial172.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T18:32:10.575 INFO:teuthology.orchestra.run.trial172.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-24T18:32:10.575 INFO:teuthology.orchestra.run.trial172.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:32:10.575 INFO:teuthology.orchestra.run.trial172.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-24T18:32:10.645 INFO:teuthology.orchestra.run.trial172.stderr: 81.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-24T18:32:10.646 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-24T18:32:10.663 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-24T18:32:10.664 DEBUG:teuthology.orchestra.run.trial172:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-24T18:32:10.695 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-24T18:32:10.711 DEBUG:teuthology.orchestra.run.trial172:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:32:10.746 INFO:teuthology.orchestra.run.trial172.stdout:kernel.core_pattern = core 2026-02-24T18:32:10.755 DEBUG:teuthology.orchestra.run.trial172:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:32:10.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:32:10.802 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-24T18:32:10.818 INFO:teuthology.task.internal:Transferring archived files... 2026-02-24T18:32:10.821 DEBUG:teuthology.misc:Transferring archived files from trial172:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:03:54-rados-wip-yuri8-testing-2026-02-23-2128-distro-default-trial/68602/remote/trial172 2026-02-24T18:32:10.823 DEBUG:teuthology.orchestra.run.trial172:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-24T19:31:42.107 INFO:teuthology.task.internal:Removing archive directory... 2026-02-24T19:31:42.113 DEBUG:teuthology.orchestra.run.trial172:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-24T19:31:45.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-24T19:31:45.864 INFO:teuthology.task.internal:Not uploading archives. 2026-02-24T19:31:45.865 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-24T19:31:45.880 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-24T19:31:45.880 DEBUG:teuthology.orchestra.run.trial172:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-24T19:31:45.886 INFO:teuthology.orchestra.run.trial172.stdout: 1311292 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 24 19:31 /home/ubuntu/cephtest 2026-02-24T19:31:45.886 INFO:teuthology.orchestra.run.trial172.stdout: 1321443 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 24 18:07 /home/ubuntu/cephtest/mnt.0 2026-02-24T19:31:45.886 INFO:teuthology.orchestra.run.trial172.stdout: 1321444 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 24 18:18 /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T19:31:45.886 INFO:teuthology.orchestra.run.trial172.stdout: 1337402 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 24 18:18 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T19:31:45.886 INFO:teuthology.orchestra.run.trial172.stdout: 1337403 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Feb 24 18:32 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td 2026-02-24T19:31:45.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T19:31:45.887 INFO:teuthology.orchestra.run.trial172.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2026-02-24T19:31:45.888 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial172 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-24T19:31:45.892 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-24T19:31:45.908 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-24T19:31:45.926 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on trial172 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-24T19:31:45.932 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} duration: 1653.4464883804321 failure_reason: 'Command failed (workunit test osd/osd-bluefs-volume-ops.sh) on trial172 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=618c318a7a0b287ab5aeff207b7bf1d73ef45044 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh''' flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-24T19:31:45.933 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-24T19:31:46.068 INFO:teuthology.run:FAIL