2026-02-17T21:11:06.373 INFO:root:teuthology version: 0.0.1.dev31+g5f66ecfb3 2026-02-17T21:11:06.388 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T21:11:06.466 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial/53907 branch: wip-yuri6-testing-2026-02-17-1732-squid description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} email: null first_in_suite: false flavor: default job_id: '53907' kernel: branch: distro kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-yuri6-testing-2026-02-17-1732-squid ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: d855f53b89fdcec760fd9232a5fb55ed4fb111a1 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: d855f53b89fdcec760fd9232a5fb55ed4fb111a1 workunit: branch: wip-yuri6-testing-2026-02-17-1732-squid sha1: d855f53b89fdcec760fd9232a5fb55ed4fb111a1 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 2987 sha1: d855f53b89fdcec760fd9232a5fb55ed4fb111a1 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri6-testing-2026-02-17-1732-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d855f53b89fdcec760fd9232a5fb55ed4fb111a1/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d855f53b89fdcec760fd9232a5fb55ed4fb111a1 targets: trial031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFVFzdf6OQMmQk41Tv6EyDMfp7Xq4o06S8ru2uWSq0lh7XsQSPYabGgzhpBdkqF7tDTChu36tIu/StjvfIxpc6U= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - osd teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: reimage-ssh-down teuthology_sha1: 5f66ecfb34c0370410e78b3ee641753d19da653b timestamp: 2026-02-17_20:43:43 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3682 2026-02-17T21:11:06.466 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d855f53b89fdcec760fd9232a5fb55ed4fb111a1/qa; will attempt to use it 2026-02-17T21:11:06.466 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d855f53b89fdcec760fd9232a5fb55ed4fb111a1/qa/tasks 2026-02-17T21:11:06.466 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-17T21:11:06.468 INFO:teuthology.task.internal:Checking packages... 2026-02-17T21:11:06.468 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd855f53b89fdcec760fd9232a5fb55ed4fb111a1' 2026-02-17T21:11:06.468 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-17T21:11:06.468 INFO:teuthology.packaging:ref: None 2026-02-17T21:11:06.468 INFO:teuthology.packaging:tag: None 2026-02-17T21:11:06.468 INFO:teuthology.packaging:branch: wip-yuri6-testing-2026-02-17-1732-squid 2026-02-17T21:11:06.468 INFO:teuthology.packaging:sha1: d855f53b89fdcec760fd9232a5fb55ed4fb111a1 2026-02-17T21:11:06.469 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-yuri6-testing-2026-02-17-1732-squid 2026-02-17T21:11:06.583 INFO:teuthology.task.internal:Found packages for ceph version 19.2.3-679-gd855f53b-1jammy 2026-02-17T21:11:06.585 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-17T21:11:06.597 INFO:teuthology.task.internal:no buildpackages task found 2026-02-17T21:11:06.597 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-17T21:11:06.612 INFO:teuthology.task.internal:Saving configuration 2026-02-17T21:11:06.619 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-17T21:11:06.632 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-17T21:11:06.712 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial031.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial/53907', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-17 21:07:37.555052', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIN9sReM/5okC/G48He0LeTDcMEZqp12rwpFEbyfuUA6S'} 2026-02-17T21:11:06.712 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-17T21:11:06.724 INFO:teuthology.task.internal:roles: ubuntu@trial031.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-17T21:11:06.724 INFO:teuthology.run_tasks:Running task console_log... 2026-02-17T21:11:06.809 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7794c43eb380>, signals=[15]) 2026-02-17T21:11:06.809 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-17T21:11:06.821 INFO:teuthology.task.internal:Opening connections... 2026-02-17T21:11:06.821 DEBUG:teuthology.task.internal:connecting to ubuntu@trial031.front.sepia.ceph.com 2026-02-17T21:11:06.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T21:11:06.884 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-17T21:11:06.897 DEBUG:teuthology.orchestra.run.trial031:> uname -m 2026-02-17T21:11:06.962 INFO:teuthology.orchestra.run.trial031.stdout:x86_64 2026-02-17T21:11:06.963 DEBUG:teuthology.orchestra.run.trial031:> cat /etc/os-release 2026-02-17T21:11:07.009 INFO:teuthology.orchestra.run.trial031.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-17T21:11:07.009 INFO:teuthology.orchestra.run.trial031.stdout:NAME="Ubuntu" 2026-02-17T21:11:07.009 INFO:teuthology.orchestra.run.trial031.stdout:VERSION_ID="22.04" 2026-02-17T21:11:07.009 INFO:teuthology.orchestra.run.trial031.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-17T21:11:07.009 INFO:teuthology.orchestra.run.trial031.stdout:VERSION_CODENAME=jammy 2026-02-17T21:11:07.009 INFO:teuthology.orchestra.run.trial031.stdout:ID=ubuntu 2026-02-17T21:11:07.009 INFO:teuthology.orchestra.run.trial031.stdout:ID_LIKE=debian 2026-02-17T21:11:07.009 INFO:teuthology.orchestra.run.trial031.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-17T21:11:07.009 INFO:teuthology.orchestra.run.trial031.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-17T21:11:07.009 INFO:teuthology.orchestra.run.trial031.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-17T21:11:07.009 INFO:teuthology.orchestra.run.trial031.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-17T21:11:07.009 INFO:teuthology.orchestra.run.trial031.stdout:UBUNTU_CODENAME=jammy 2026-02-17T21:11:07.010 INFO:teuthology.lock.ops:Updating trial031.front.sepia.ceph.com on lock server 2026-02-17T21:11:07.098 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-17T21:11:07.113 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-17T21:11:07.126 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-17T21:11:07.126 DEBUG:teuthology.orchestra.run.trial031:> test '!' -e /home/ubuntu/cephtest 2026-02-17T21:11:07.129 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-17T21:11:07.140 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-17T21:11:07.140 DEBUG:teuthology.orchestra.run.trial031:> test -z $(ls -A /var/lib/ceph) 2026-02-17T21:11:07.177 INFO:teuthology.orchestra.run.trial031.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-17T21:11:07.178 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-17T21:11:07.281 INFO:teuthology.run_tasks:Running task kernel... 2026-02-17T21:11:07.303 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-17T21:11:07.303 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-17T21:11:07.303 DEBUG:teuthology.orchestra.run.trial031:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-17T21:11:07.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T21:11:07.307 DEBUG:teuthology.orchestra.run.trial031:> uname -r 2026-02-17T21:11:07.357 INFO:teuthology.orchestra.run.trial031.stdout:5.15.0-164-generic 2026-02-17T21:11:07.357 INFO:teuthology.task.kernel:Running kernel on trial031: 5.15.0-164-generic 2026-02-17T21:11:07.357 DEBUG:teuthology.orchestra.run.trial031:> sudo apt-get clean 2026-02-17T21:11:07.419 DEBUG:teuthology.orchestra.run.trial031:> sudo apt-get update 2026-02-17T21:11:07.618 INFO:teuthology.orchestra.run.trial031.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-17T21:11:07.626 INFO:teuthology.orchestra.run.trial031.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-17T21:11:07.667 INFO:teuthology.orchestra.run.trial031.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-17T21:11:07.679 INFO:teuthology.orchestra.run.trial031.stdout:Hit:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-17T21:11:07.764 INFO:teuthology.orchestra.run.trial031.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3237 kB] 2026-02-17T21:11:07.841 INFO:teuthology.orchestra.run.trial031.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [493 kB] 2026-02-17T21:11:07.844 INFO:teuthology.orchestra.run.trial031.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1255 kB] 2026-02-17T21:11:09.312 INFO:teuthology.orchestra.run.trial031.stdout:Fetched 5114 kB in 1s (8111 kB/s) 2026-02-17T21:11:10.005 INFO:teuthology.orchestra.run.trial031.stdout:Reading package lists... 2026-02-17T21:11:10.019 DEBUG:teuthology.orchestra.run.trial031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-17T21:11:10.050 INFO:teuthology.orchestra.run.trial031.stdout:Reading package lists... 2026-02-17T21:11:10.160 INFO:teuthology.orchestra.run.trial031.stdout:Building dependency tree... 2026-02-17T21:11:10.161 INFO:teuthology.orchestra.run.trial031.stdout:Reading state information... 2026-02-17T21:11:10.247 INFO:teuthology.orchestra.run.trial031.stdout:The following packages were automatically installed and are no longer required: 2026-02-17T21:11:10.247 INFO:teuthology.orchestra.run.trial031.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-17T21:11:10.247 INFO:teuthology.orchestra.run.trial031.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-17T21:11:10.247 INFO:teuthology.orchestra.run.trial031.stdout: sg3-utils-udev 2026-02-17T21:11:10.247 INFO:teuthology.orchestra.run.trial031.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-17T21:11:10.247 INFO:teuthology.orchestra.run.trial031.stdout:The following additional packages will be installed: 2026-02-17T21:11:10.247 INFO:teuthology.orchestra.run.trial031.stdout: linux-generic linux-headers-5.15.0-170 linux-headers-5.15.0-170-generic 2026-02-17T21:11:10.247 INFO:teuthology.orchestra.run.trial031.stdout: linux-headers-generic linux-image-5.15.0-170-generic 2026-02-17T21:11:10.247 INFO:teuthology.orchestra.run.trial031.stdout: linux-modules-5.15.0-170-generic linux-modules-extra-5.15.0-170-generic 2026-02-17T21:11:10.248 INFO:teuthology.orchestra.run.trial031.stdout:Suggested packages: 2026-02-17T21:11:10.248 INFO:teuthology.orchestra.run.trial031.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2026-02-17T21:11:10.260 INFO:teuthology.orchestra.run.trial031.stdout:The following NEW packages will be installed: 2026-02-17T21:11:10.260 INFO:teuthology.orchestra.run.trial031.stdout: linux-headers-5.15.0-170 linux-headers-5.15.0-170-generic 2026-02-17T21:11:10.260 INFO:teuthology.orchestra.run.trial031.stdout: linux-image-5.15.0-170-generic linux-modules-5.15.0-170-generic 2026-02-17T21:11:10.261 INFO:teuthology.orchestra.run.trial031.stdout: linux-modules-extra-5.15.0-170-generic 2026-02-17T21:11:10.261 INFO:teuthology.orchestra.run.trial031.stdout:The following packages will be upgraded: 2026-02-17T21:11:10.261 INFO:teuthology.orchestra.run.trial031.stdout: linux-generic linux-headers-generic linux-image-generic 2026-02-17T21:11:10.605 INFO:teuthology.orchestra.run.trial031.stdout:3 upgraded, 5 newly installed, 0 to remove and 53 not upgraded. 2026-02-17T21:11:10.605 INFO:teuthology.orchestra.run.trial031.stdout:Need to get 113 MB of archives. 2026-02-17T21:11:10.606 INFO:teuthology.orchestra.run.trial031.stdout:After this operation, 585 MB of additional disk space will be used. 2026-02-17T21:11:10.606 INFO:teuthology.orchestra.run.trial031.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-170-generic amd64 5.15.0-170.180 [22.7 MB] 2026-02-17T21:11:11.697 INFO:teuthology.orchestra.run.trial031.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-170-generic amd64 5.15.0-170.180 [11.6 MB] 2026-02-17T21:11:11.967 INFO:teuthology.orchestra.run.trial031.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-170-generic amd64 5.15.0-170.180 [64.0 MB] 2026-02-17T21:11:13.459 INFO:teuthology.orchestra.run.trial031.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.170.159 [1654 B] 2026-02-17T21:11:13.459 INFO:teuthology.orchestra.run.trial031.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.170.159 [2474 B] 2026-02-17T21:11:13.459 INFO:teuthology.orchestra.run.trial031.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-170 all 5.15.0-170.180 [12.4 MB] 2026-02-17T21:11:13.737 INFO:teuthology.orchestra.run.trial031.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-170-generic amd64 5.15.0-170.180 [2828 kB] 2026-02-17T21:11:13.802 INFO:teuthology.orchestra.run.trial031.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.170.159 [2316 B] 2026-02-17T21:11:14.003 INFO:teuthology.orchestra.run.trial031.stdout:Fetched 113 MB in 4s (32.0 MB/s) 2026-02-17T21:11:14.024 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package linux-modules-5.15.0-170-generic. 2026-02-17T21:11:14.188 INFO:teuthology.orchestra.run.trial031.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 131954 files and directories currently installed.) 2026-02-17T21:11:14.190 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../0-linux-modules-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T21:11:14.191 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking linux-modules-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T21:11:14.538 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package linux-image-5.15.0-170-generic. 2026-02-17T21:11:14.544 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../1-linux-image-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T21:11:14.567 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T21:11:14.607 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-170-generic. 2026-02-17T21:11:14.614 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T21:11:14.615 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking linux-modules-extra-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T21:11:15.720 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../3-linux-generic_5.15.0.170.159_amd64.deb ... 2026-02-17T21:11:15.722 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking linux-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-17T21:11:15.743 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.170.159_amd64.deb ... 2026-02-17T21:11:15.744 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking linux-image-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-17T21:11:15.760 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package linux-headers-5.15.0-170. 2026-02-17T21:11:15.767 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../5-linux-headers-5.15.0-170_5.15.0-170.180_all.deb ... 2026-02-17T21:11:15.768 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking linux-headers-5.15.0-170 (5.15.0-170.180) ... 2026-02-17T21:11:17.164 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package linux-headers-5.15.0-170-generic. 2026-02-17T21:11:17.176 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../6-linux-headers-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T21:11:17.176 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking linux-headers-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T21:11:17.758 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.170.159_amd64.deb ... 2026-02-17T21:11:17.759 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking linux-headers-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-17T21:11:17.784 INFO:teuthology.orchestra.run.trial031.stdout:Setting up linux-headers-5.15.0-170 (5.15.0-170.180) ... 2026-02-17T21:11:17.785 INFO:teuthology.orchestra.run.trial031.stdout:Setting up linux-headers-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T21:11:17.786 INFO:teuthology.orchestra.run.trial031.stdout:Setting up linux-headers-generic (5.15.0.170.159) ... 2026-02-17T21:11:17.788 INFO:teuthology.orchestra.run.trial031.stdout:Setting up linux-modules-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T21:11:18.567 INFO:teuthology.orchestra.run.trial031.stdout:Setting up linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T21:11:19.388 INFO:teuthology.orchestra.run.trial031.stdout:I: /boot/vmlinuz is now a symlink to vmlinuz-5.15.0-170-generic 2026-02-17T21:11:19.388 INFO:teuthology.orchestra.run.trial031.stdout:I: /boot/initrd.img is now a symlink to initrd.img-5.15.0-170-generic 2026-02-17T21:11:19.395 INFO:teuthology.orchestra.run.trial031.stdout:Setting up linux-modules-extra-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T21:11:20.187 INFO:teuthology.orchestra.run.trial031.stdout:Setting up linux-image-generic (5.15.0.170.159) ... 2026-02-17T21:11:20.188 INFO:teuthology.orchestra.run.trial031.stdout:Setting up linux-generic (5.15.0.170.159) ... 2026-02-17T21:11:20.190 INFO:teuthology.orchestra.run.trial031.stdout:Processing triggers for linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T21:11:20.195 INFO:teuthology.orchestra.run.trial031.stdout:/etc/kernel/postinst.d/initramfs-tools: 2026-02-17T21:11:20.195 INFO:teuthology.orchestra.run.trial031.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-170-generic 2026-02-17T21:11:27.285 INFO:teuthology.orchestra.run.trial031.stdout:W: Possible missing firmware /lib/firmware/ast_dp501_fw.bin for module ast 2026-02-17T21:11:33.808 INFO:teuthology.orchestra.run.trial031.stdout:/etc/kernel/postinst.d/zz-update-grub: 2026-02-17T21:11:33.808 INFO:teuthology.orchestra.run.trial031.stdout:Sourcing file `/etc/default/grub' 2026-02-17T21:11:33.831 INFO:teuthology.orchestra.run.trial031.stdout:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-17T21:11:33.831 INFO:teuthology.orchestra.run.trial031.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-17T21:11:33.834 INFO:teuthology.orchestra.run.trial031.stdout:Generating grub configuration file ... 2026-02-17T21:11:33.915 INFO:teuthology.orchestra.run.trial031.stdout:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-17T21:11:33.921 INFO:teuthology.orchestra.run.trial031.stdout:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-17T21:11:34.093 INFO:teuthology.orchestra.run.trial031.stdout:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-17T21:11:34.098 INFO:teuthology.orchestra.run.trial031.stdout:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-17T21:11:34.168 INFO:teuthology.orchestra.run.trial031.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-17T21:11:34.168 INFO:teuthology.orchestra.run.trial031.stdout:Systems on them will not be added to the GRUB boot configuration. 2026-02-17T21:11:34.169 INFO:teuthology.orchestra.run.trial031.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-17T21:11:34.171 INFO:teuthology.orchestra.run.trial031.stdout:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T21:11:34.180 INFO:teuthology.orchestra.run.trial031.stdout:done 2026-02-17T21:11:34.598 INFO:teuthology.orchestra.run.trial031.stdout: 2026-02-17T21:11:34.598 INFO:teuthology.orchestra.run.trial031.stdout:Pending kernel upgrade! 2026-02-17T21:11:34.598 INFO:teuthology.orchestra.run.trial031.stdout: 2026-02-17T21:11:34.598 INFO:teuthology.orchestra.run.trial031.stdout:Running kernel version: 2026-02-17T21:11:34.599 INFO:teuthology.orchestra.run.trial031.stdout: 5.15.0-164-generic 2026-02-17T21:11:34.599 INFO:teuthology.orchestra.run.trial031.stdout: 2026-02-17T21:11:34.599 INFO:teuthology.orchestra.run.trial031.stdout:Diagnostics: 2026-02-17T21:11:34.599 INFO:teuthology.orchestra.run.trial031.stdout: The currently running kernel version is not the expected kernel version 5.15.0-170-generic. 2026-02-17T21:11:34.599 INFO:teuthology.orchestra.run.trial031.stdout: 2026-02-17T21:11:34.599 INFO:teuthology.orchestra.run.trial031.stdout:Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. [Return] 2026-02-17T21:11:34.599 INFO:teuthology.orchestra.run.trial031.stdout: 2026-02-17T21:11:34.599 INFO:teuthology.orchestra.run.trial031.stdout:The processor microcode seems to be up-to-date. 2026-02-17T21:11:34.599 INFO:teuthology.orchestra.run.trial031.stdout: 2026-02-17T21:11:34.599 INFO:teuthology.orchestra.run.trial031.stdout:No services need to be restarted. 2026-02-17T21:11:34.599 INFO:teuthology.orchestra.run.trial031.stdout: 2026-02-17T21:11:34.599 INFO:teuthology.orchestra.run.trial031.stdout:No containers need to be restarted. 2026-02-17T21:11:34.600 INFO:teuthology.orchestra.run.trial031.stdout: 2026-02-17T21:11:34.600 INFO:teuthology.orchestra.run.trial031.stdout:No user sessions are running outdated binaries. 2026-02-17T21:11:34.600 INFO:teuthology.orchestra.run.trial031.stdout: 2026-02-17T21:11:34.600 INFO:teuthology.orchestra.run.trial031.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-17T21:11:35.291 DEBUG:teuthology.orchestra.run.trial031:> dpkg -s linux-image-generic 2026-02-17T21:11:35.301 INFO:teuthology.orchestra.run.trial031.stdout:Package: linux-image-generic 2026-02-17T21:11:35.301 INFO:teuthology.orchestra.run.trial031.stdout:Status: install ok installed 2026-02-17T21:11:35.301 INFO:teuthology.orchestra.run.trial031.stdout:Priority: optional 2026-02-17T21:11:35.301 INFO:teuthology.orchestra.run.trial031.stdout:Section: kernel 2026-02-17T21:11:35.302 INFO:teuthology.orchestra.run.trial031.stdout:Installed-Size: 21 2026-02-17T21:11:35.302 INFO:teuthology.orchestra.run.trial031.stdout:Maintainer: Ubuntu Kernel Team 2026-02-17T21:11:35.302 INFO:teuthology.orchestra.run.trial031.stdout:Architecture: amd64 2026-02-17T21:11:35.302 INFO:teuthology.orchestra.run.trial031.stdout:Source: linux-meta 2026-02-17T21:11:35.302 INFO:teuthology.orchestra.run.trial031.stdout:Version: 5.15.0.170.159 2026-02-17T21:11:35.302 INFO:teuthology.orchestra.run.trial031.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-17T21:11:35.302 INFO:teuthology.orchestra.run.trial031.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-17T21:11:35.302 INFO:teuthology.orchestra.run.trial031.stdout:Recommends: thermald 2026-02-17T21:11:35.302 INFO:teuthology.orchestra.run.trial031.stdout:Description: Generic Linux kernel image 2026-02-17T21:11:35.302 INFO:teuthology.orchestra.run.trial031.stdout: This package will always depend on the latest generic kernel image 2026-02-17T21:11:35.302 INFO:teuthology.orchestra.run.trial031.stdout: available. 2026-02-17T21:11:35.303 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-164-generic Expected: 5.15.0-170-generic 2026-02-17T21:11:35.303 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-17T21:11:35.303 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-17T21:11:35.303 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-17T21:11:35.303 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial031.front.sepia.ceph.com, path=None, version=distro) 2026-02-17T21:11:35.303 DEBUG:teuthology.orchestra.run.trial031:> sudo apt-get clean 2026-02-17T21:11:35.388 DEBUG:teuthology.orchestra.run.trial031:> sudo apt-get update 2026-02-17T21:11:35.590 INFO:teuthology.orchestra.run.trial031.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-17T21:11:35.595 INFO:teuthology.orchestra.run.trial031.stdout:Hit:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-17T21:11:35.606 INFO:teuthology.orchestra.run.trial031.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-17T21:11:35.615 INFO:teuthology.orchestra.run.trial031.stdout:Hit:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-17T21:11:36.447 INFO:teuthology.orchestra.run.trial031.stdout:Reading package lists... 2026-02-17T21:11:36.461 DEBUG:teuthology.orchestra.run.trial031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-17T21:11:36.494 INFO:teuthology.orchestra.run.trial031.stdout:Reading package lists... 2026-02-17T21:11:36.605 INFO:teuthology.orchestra.run.trial031.stdout:Building dependency tree... 2026-02-17T21:11:36.605 INFO:teuthology.orchestra.run.trial031.stdout:Reading state information... 2026-02-17T21:11:36.690 INFO:teuthology.orchestra.run.trial031.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-17T21:11:36.691 INFO:teuthology.orchestra.run.trial031.stdout:linux-image-generic set to manually installed. 2026-02-17T21:11:36.691 INFO:teuthology.orchestra.run.trial031.stdout:The following packages were automatically installed and are no longer required: 2026-02-17T21:11:36.691 INFO:teuthology.orchestra.run.trial031.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-17T21:11:36.691 INFO:teuthology.orchestra.run.trial031.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-17T21:11:36.691 INFO:teuthology.orchestra.run.trial031.stdout: sg3-utils-udev 2026-02-17T21:11:36.691 INFO:teuthology.orchestra.run.trial031.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-17T21:11:36.710 INFO:teuthology.orchestra.run.trial031.stdout:0 upgraded, 0 newly installed, 0 to remove and 53 not upgraded. 2026-02-17T21:11:36.712 DEBUG:teuthology.orchestra.run.trial031:> dpkg -s linux-image-generic 2026-02-17T21:11:36.762 INFO:teuthology.orchestra.run.trial031.stdout:Package: linux-image-generic 2026-02-17T21:11:36.763 INFO:teuthology.orchestra.run.trial031.stdout:Status: install ok installed 2026-02-17T21:11:36.763 INFO:teuthology.orchestra.run.trial031.stdout:Priority: optional 2026-02-17T21:11:36.763 INFO:teuthology.orchestra.run.trial031.stdout:Section: kernel 2026-02-17T21:11:36.763 INFO:teuthology.orchestra.run.trial031.stdout:Installed-Size: 21 2026-02-17T21:11:36.763 INFO:teuthology.orchestra.run.trial031.stdout:Maintainer: Ubuntu Kernel Team 2026-02-17T21:11:36.763 INFO:teuthology.orchestra.run.trial031.stdout:Architecture: amd64 2026-02-17T21:11:36.763 INFO:teuthology.orchestra.run.trial031.stdout:Source: linux-meta 2026-02-17T21:11:36.763 INFO:teuthology.orchestra.run.trial031.stdout:Version: 5.15.0.170.159 2026-02-17T21:11:36.763 INFO:teuthology.orchestra.run.trial031.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-17T21:11:36.764 INFO:teuthology.orchestra.run.trial031.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-17T21:11:36.764 INFO:teuthology.orchestra.run.trial031.stdout:Recommends: thermald 2026-02-17T21:11:36.764 INFO:teuthology.orchestra.run.trial031.stdout:Description: Generic Linux kernel image 2026-02-17T21:11:36.764 INFO:teuthology.orchestra.run.trial031.stdout: This package will always depend on the latest generic kernel image 2026-02-17T21:11:36.764 INFO:teuthology.orchestra.run.trial031.stdout: available. 2026-02-17T21:11:36.764 DEBUG:teuthology.orchestra.run.trial031:> mktemp 2026-02-17T21:11:36.809 INFO:teuthology.orchestra.run.trial031.stdout:/tmp/tmp.gCxawnVClx 2026-02-17T21:11:36.809 DEBUG:teuthology.orchestra.run.trial031:> sudo cp /boot/grub/grub.cfg /tmp/tmp.gCxawnVClx 2026-02-17T21:11:36.861 DEBUG:teuthology.orchestra.run.trial031:> sudo chmod 0666 /tmp/tmp.gCxawnVClx 2026-02-17T21:11:36.959 DEBUG:teuthology.orchestra.remote:trial031:/tmp/tmp.gCxawnVClx is 7KB 2026-02-17T21:11:36.967 DEBUG:teuthology.orchestra.run.trial031:> rm -fr /tmp/tmp.gCxawnVClx 2026-02-17T21:11:36.972 DEBUG:teuthology.orchestra.run.trial031:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2026-02-17T21:11:37.020 DEBUG:teuthology.orchestra.run.trial031:> set -ex 2026-02-17T21:11:37.021 DEBUG:teuthology.orchestra.run.trial031:> sudo dd of=/etc/grub.d/01_ceph_kernel 2026-02-17T21:11:37.021 DEBUG:teuthology.orchestra.run.trial031:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2026-02-17T21:11:37.075 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-170-generic 2026-02-17T21:11:37.075 DEBUG:teuthology.orchestra.run.trial031:> sudo update-grub 2026-02-17T21:11:37.320 INFO:teuthology.orchestra.run.trial031.stderr:Sourcing file `/etc/default/grub' 2026-02-17T21:11:37.339 INFO:teuthology.orchestra.run.trial031.stderr:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-17T21:11:37.340 INFO:teuthology.orchestra.run.trial031.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-17T21:11:37.342 INFO:teuthology.orchestra.run.trial031.stderr:Generating grub configuration file ... 2026-02-17T21:11:37.425 INFO:teuthology.orchestra.run.trial031.stderr:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-17T21:11:37.431 INFO:teuthology.orchestra.run.trial031.stderr:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-17T21:11:37.605 INFO:teuthology.orchestra.run.trial031.stderr:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-17T21:11:37.610 INFO:teuthology.orchestra.run.trial031.stderr:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-17T21:11:37.681 INFO:teuthology.orchestra.run.trial031.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-17T21:11:37.681 INFO:teuthology.orchestra.run.trial031.stderr:Systems on them will not be added to the GRUB boot configuration. 2026-02-17T21:11:37.681 INFO:teuthology.orchestra.run.trial031.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-17T21:11:37.683 INFO:teuthology.orchestra.run.trial031.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T21:11:37.691 INFO:teuthology.orchestra.run.trial031.stderr:done 2026-02-17T21:11:37.692 DEBUG:teuthology.orchestra.run.trial031:> sudo shutdown -r now 2026-02-17T21:12:07.722 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-17T21:12:07.723 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial031.front.sepia.ceph.com' 2026-02-17T21:12:07.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T21:12:22.017 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.31 2026-02-17T21:12:31.026 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial031.front.sepia.ceph.com' 2026-02-17T21:12:31.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T21:13:06.812 DEBUG:teuthology.orchestra.run.trial031:> true 2026-02-17T21:13:07.208 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial031.front.sepia.ceph.com' 2026-02-17T21:13:07.208 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-170-generic"... 2026-02-17T21:13:07.208 DEBUG:teuthology.orchestra.run.trial031:> uname -r 2026-02-17T21:13:07.252 INFO:teuthology.orchestra.run.trial031.stdout:5.15.0-170-generic 2026-02-17T21:13:07.253 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-170-generic vs 5.15.0-170-generic 2026-02-17T21:13:07.253 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-17T21:13:07.253 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2026-02-17T21:13:08.254 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-17T21:13:08.254 DEBUG:teuthology.orchestra.run.trial031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-17T21:13:08.267 INFO:teuthology.orchestra.run.trial031.stdout:ttyS1 2026-02-17T21:13:08.287 DEBUG:teuthology.parallel:result is None 2026-02-17T21:13:08.288 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-17T21:13:08.299 INFO:teuthology.task.internal:Creating test directory... 2026-02-17T21:13:08.299 DEBUG:teuthology.orchestra.run.trial031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-17T21:13:08.333 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-17T21:13:08.345 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-17T21:13:08.359 INFO:teuthology.task.internal:Creating archive directory... 2026-02-17T21:13:08.359 DEBUG:teuthology.orchestra.run.trial031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-17T21:13:08.383 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-17T21:13:08.395 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-17T21:13:08.396 DEBUG:teuthology.orchestra.run.trial031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-17T21:13:08.430 INFO:teuthology.orchestra.run.trial031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:13:08.436 INFO:teuthology.orchestra.run.trial031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:13:08.437 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-17T21:13:08.450 INFO:teuthology.task.internal:Configuring sudo... 2026-02-17T21:13:08.451 DEBUG:teuthology.orchestra.run.trial031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-17T21:13:08.489 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-17T21:13:08.507 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-17T21:13:08.507 DEBUG:teuthology.orchestra.run.trial031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-17T21:13:08.537 DEBUG:teuthology.orchestra.run.trial031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T21:13:08.581 DEBUG:teuthology.orchestra.run.trial031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T21:13:08.625 DEBUG:teuthology.orchestra.run.trial031:> set -ex 2026-02-17T21:13:08.625 DEBUG:teuthology.orchestra.run.trial031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-17T21:13:08.673 DEBUG:teuthology.orchestra.run.trial031:> sudo service rsyslog restart 2026-02-17T21:13:08.734 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-17T21:13:08.747 INFO:teuthology.task.internal:Starting timer... 2026-02-17T21:13:08.747 INFO:teuthology.run_tasks:Running task pcp... 2026-02-17T21:13:08.770 INFO:teuthology.run_tasks:Running task selinux... 2026-02-17T21:13:08.789 DEBUG:teuthology.task.selinux:Excluding trial031: OS 'ubuntu' does not support SELinux 2026-02-17T21:13:08.789 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-17T21:13:08.790 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-17T21:13:08.790 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-17T21:13:08.790 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-17T21:13:08.808 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-17T21:13:08.820 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-17T21:13:08.822 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial031.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-17T21:14:08.483 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial031.front.sepia.ceph.com')] 2026-02-17T21:14:08.484 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial031.front.sepia.ceph.com' 2026-02-17T21:14:08.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T21:14:08.554 DEBUG:teuthology.orchestra.run.trial031:> true 2026-02-17T21:14:08.809 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial031.front.sepia.ceph.com' 2026-02-17T21:14:08.809 INFO:teuthology.run_tasks:Running task clock... 2026-02-17T21:14:08.827 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-17T21:14:08.827 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-17T21:14:08.828 DEBUG:teuthology.orchestra.run.trial031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T21:14:08.870 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-17T21:14:08.870 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: Command line: ntpd -gq 2026-02-17T21:14:08.870 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: ---------------------------------------------------- 2026-02-17T21:14:08.870 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: ntp-4 is maintained by Network Time Foundation, 2026-02-17T21:14:08.870 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-17T21:14:08.870 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: corporation. Support and training for ntp-4 are 2026-02-17T21:14:08.870 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: available at https://www.nwtime.org/support 2026-02-17T21:14:08.870 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: ---------------------------------------------------- 2026-02-17T21:14:08.871 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: proto: precision = 0.030 usec (-25) 2026-02-17T21:14:08.871 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: basedate set to 2022-02-04 2026-02-17T21:14:08.871 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: gps base set to 2022-02-06 (week 2196) 2026-02-17T21:14:08.871 INFO:teuthology.orchestra.run.trial031.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-17T21:14:08.872 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-17T21:14:08.872 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: restrict ::: KOD does nothing without LIMITED. 2026-02-17T21:14:08.872 INFO:teuthology.orchestra.run.trial031.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-17T21:14:08.873 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: Listen and drop on 0 v6wildcard [::]:123 2026-02-17T21:14:08.873 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-17T21:14:08.873 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: Listen normally on 2 lo 127.0.0.1:123 2026-02-17T21:14:08.873 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: Listen normally on 3 enp1s0f0 10.20.193.31:123 2026-02-17T21:14:08.873 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: Listen normally on 4 lo [::1]:123 2026-02-17T21:14:08.873 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:6236%2]:123 2026-02-17T21:14:08.873 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:09 ntpd[4889]: Listening on routing socket on fd #22 for interface updates 2026-02-17T21:14:09.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:10 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:09.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:10 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:09.902 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:10 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:09.902 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:10 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:09.902 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:10 ntpd[4889]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T21:14:09.902 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:10 ntpd[4889]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T21:14:11.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:11.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:11.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:11.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:11.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:11.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:11.901 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:11.901 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:11.902 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T21:14:11.902 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T21:14:11.940 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:11.940 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:11.940 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T21:14:11.941 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T21:14:11.980 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:11.981 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:11.981 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T21:14:11.981 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:12 ntpd[4889]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T21:14:13.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:13.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:13.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:13.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:13.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:13.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:13.901 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:13.901 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:13.901 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T21:14:13.901 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T21:14:13.940 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:13.940 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:13.940 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T21:14:13.940 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T21:14:13.980 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:13.980 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:13.980 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T21:14:13.980 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:14 ntpd[4889]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T21:14:14.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:15 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:14.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:15 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:14.938 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:15 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:14.938 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:15 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:14.938 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:15 ntpd[4889]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T21:14:14.939 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:15 ntpd[4889]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T21:14:15.870 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:16 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:15.871 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:16 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:15.871 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:16 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:15.871 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:16 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:15.871 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:16 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:15.871 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:16 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:15.901 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:16 ntpd[4889]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T21:14:15.902 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:16 ntpd[4889]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T21:14:15.902 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:16 ntpd[4889]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T21:14:15.902 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:16 ntpd[4889]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T21:14:15.902 INFO:teuthology.orchestra.run.trial031.stdout:17 Feb 21:14:15 ntpd[4889]: ntpd: time set -0.813231 s 2026-02-17T21:14:15.903 INFO:teuthology.orchestra.run.trial031.stdout:ntpd: time set -0.813231s 2026-02-17T21:14:15.903 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:15 ntpd[4889]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-17T21:14:15.904 INFO:teuthology.orchestra.run.trial031.stderr:17 Feb 21:14:15 ntpd[4889]: can't open /var/log/ntpstats/loopstats.20260217: Permission denied 2026-02-17T21:14:15.971 INFO:teuthology.orchestra.run.trial031.stdout: remote refid st t when poll reach delay offset jitter 2026-02-17T21:14:15.971 INFO:teuthology.orchestra.run.trial031.stdout:============================================================================== 2026-02-17T21:14:15.972 INFO:teuthology.orchestra.run.trial031.stdout: 5.78.90.46 (jmp .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-17T21:14:15.972 INFO:teuthology.orchestra.run.trial031.stdout: triton.ellipse. .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-17T21:14:15.972 INFO:teuthology.orchestra.run.trial031.stdout: 172-235-154-118 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-17T21:14:15.972 INFO:teuthology.orchestra.run.trial031.stdout: as393746.mci.tr .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-17T21:14:15.973 INFO:teuthology.run_tasks:Running task install... 2026-02-17T21:14:15.988 DEBUG:teuthology.task.install:project ceph 2026-02-17T21:14:15.988 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd855f53b89fdcec760fd9232a5fb55ed4fb111a1'}} 2026-02-17T21:14:15.988 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd855f53b89fdcec760fd9232a5fb55ed4fb111a1'} 2026-02-17T21:14:15.988 INFO:teuthology.task.install:Using flavor: default 2026-02-17T21:14:15.999 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-17T21:14:15.999 INFO:teuthology.task.install:extra packages: [] 2026-02-17T21:14:16.000 DEBUG:teuthology.orchestra.run.trial031:> sudo apt-key list | grep Ceph 2026-02-17T21:14:16.068 INFO:teuthology.orchestra.run.trial031.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-17T21:14:16.098 INFO:teuthology.orchestra.run.trial031.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-17T21:14:16.099 INFO:teuthology.orchestra.run.trial031.stdout:uid [ unknown] Ceph.com (release key) 2026-02-17T21:14:16.099 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-17T21:14:16.100 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d855f53b89fdcec760fd9232a5fb55ed4fb111a1 2026-02-17T21:14:16.288 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default/ 2026-02-17T21:14:16.288 INFO:teuthology.task.install.deb:Package version is 19.2.3-679-gd855f53b-1jammy 2026-02-17T21:14:16.564 DEBUG:teuthology.orchestra.run.trial031:> set -ex 2026-02-17T21:14:16.564 DEBUG:teuthology.orchestra.run.trial031:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-17T21:14:16.574 DEBUG:teuthology.orchestra.run.trial031:> sudo apt-get update 2026-02-17T21:14:16.851 INFO:teuthology.orchestra.run.trial031.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy InRelease 2026-02-17T21:14:16.894 INFO:teuthology.orchestra.run.trial031.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy Release [7662 B] 2026-02-17T21:14:16.938 INFO:teuthology.orchestra.run.trial031.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-17T21:14:16.981 INFO:teuthology.orchestra.run.trial031.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2026-02-17T21:14:17.086 INFO:teuthology.orchestra.run.trial031.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-17T21:14:17.153 INFO:teuthology.orchestra.run.trial031.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-17T21:14:17.225 INFO:teuthology.orchestra.run.trial031.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-17T21:14:17.297 INFO:teuthology.orchestra.run.trial031.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-17T21:14:17.442 INFO:teuthology.orchestra.run.trial031.stdout:Fetched 25.8 kB in 1s (36.4 kB/s) 2026-02-17T21:14:17.981 INFO:teuthology.orchestra.run.trial031.stdout:Reading package lists... 2026-02-17T21:14:17.995 DEBUG:teuthology.orchestra.run.trial031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.2.3-679-gd855f53b-1jammy cephadm=19.2.3-679-gd855f53b-1jammy ceph-mds=19.2.3-679-gd855f53b-1jammy ceph-mgr=19.2.3-679-gd855f53b-1jammy ceph-common=19.2.3-679-gd855f53b-1jammy ceph-fuse=19.2.3-679-gd855f53b-1jammy ceph-test=19.2.3-679-gd855f53b-1jammy ceph-volume=19.2.3-679-gd855f53b-1jammy radosgw=19.2.3-679-gd855f53b-1jammy python3-rados=19.2.3-679-gd855f53b-1jammy python3-rgw=19.2.3-679-gd855f53b-1jammy python3-cephfs=19.2.3-679-gd855f53b-1jammy python3-rbd=19.2.3-679-gd855f53b-1jammy libcephfs2=19.2.3-679-gd855f53b-1jammy libcephfs-dev=19.2.3-679-gd855f53b-1jammy librados2=19.2.3-679-gd855f53b-1jammy librbd1=19.2.3-679-gd855f53b-1jammy rbd-fuse=19.2.3-679-gd855f53b-1jammy 2026-02-17T21:14:18.028 INFO:teuthology.orchestra.run.trial031.stdout:Reading package lists... 2026-02-17T21:14:18.140 INFO:teuthology.orchestra.run.trial031.stdout:Building dependency tree... 2026-02-17T21:14:18.141 INFO:teuthology.orchestra.run.trial031.stdout:Reading state information... 2026-02-17T21:14:18.234 INFO:teuthology.orchestra.run.trial031.stdout:The following packages were automatically installed and are no longer required: 2026-02-17T21:14:18.234 INFO:teuthology.orchestra.run.trial031.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-17T21:14:18.234 INFO:teuthology.orchestra.run.trial031.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-17T21:14:18.234 INFO:teuthology.orchestra.run.trial031.stdout: sg3-utils-udev 2026-02-17T21:14:18.234 INFO:teuthology.orchestra.run.trial031.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-17T21:14:18.234 INFO:teuthology.orchestra.run.trial031.stdout:The following additional packages will be installed: 2026-02-17T21:14:18.234 INFO:teuthology.orchestra.run.trial031.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-17T21:14:18.234 INFO:teuthology.orchestra.run.trial031.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2026-02-17T21:14:18.235 INFO:teuthology.orchestra.run.trial031.stdout: libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 2026-02-17T21:14:18.235 INFO:teuthology.orchestra.run.trial031.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-17T21:14:18.235 INFO:teuthology.orchestra.run.trial031.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2026-02-17T21:14:18.235 INFO:teuthology.orchestra.run.trial031.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2026-02-17T21:14:18.235 INFO:teuthology.orchestra.run.trial031.stdout: pkg-config python-asyncssh-doc python-pastedeploy-tpl python3-asyncssh 2026-02-17T21:14:18.235 INFO:teuthology.orchestra.run.trial031.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cheroot 2026-02-17T21:14:18.235 INFO:teuthology.orchestra.run.trial031.stdout: python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-17T21:14:18.235 INFO:teuthology.orchestra.run.trial031.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-17T21:14:18.235 INFO:teuthology.orchestra.run.trial031.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-logutils 2026-02-17T21:14:18.235 INFO:teuthology.orchestra.run.trial031.stdout: python3-mako python3-natsort python3-paste python3-pastedeploy 2026-02-17T21:14:18.235 INFO:teuthology.orchestra.run.trial031.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2026-02-17T21:14:18.235 INFO:teuthology.orchestra.run.trial031.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-17T21:14:18.236 INFO:teuthology.orchestra.run.trial031.stdout: python3-pyinotify python3-pytest python3-repoze.lru 2026-02-17T21:14:18.236 INFO:teuthology.orchestra.run.trial031.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2026-02-17T21:14:18.236 INFO:teuthology.orchestra.run.trial031.stdout: python3-simplejson python3-singledispatch python3-sklearn 2026-02-17T21:14:18.236 INFO:teuthology.orchestra.run.trial031.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2026-02-17T21:14:18.236 INFO:teuthology.orchestra.run.trial031.stdout: python3-toml python3-waitress python3-wcwidth python3-webob 2026-02-17T21:14:18.236 INFO:teuthology.orchestra.run.trial031.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2026-02-17T21:14:18.236 INFO:teuthology.orchestra.run.trial031.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2026-02-17T21:14:18.236 INFO:teuthology.orchestra.run.trial031.stdout:Suggested packages: 2026-02-17T21:14:18.236 INFO:teuthology.orchestra.run.trial031.stdout: python3-influxdb readline-doc python3-beaker python-mako-doc 2026-02-17T21:14:18.236 INFO:teuthology.orchestra.run.trial031.stdout: python-natsort-doc httpd-wsgi libapache2-mod-python libapache2-mod-scgi 2026-02-17T21:14:18.236 INFO:teuthology.orchestra.run.trial031.stdout: libjs-mochikit python-pecan-doc python-psutil-doc subversion 2026-02-17T21:14:18.236 INFO:teuthology.orchestra.run.trial031.stdout: python-pygments-doc ttf-bitstream-vera python-pyinotify-doc python3-dap 2026-02-17T21:14:18.237 INFO:teuthology.orchestra.run.trial031.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2026-02-17T21:14:18.237 INFO:teuthology.orchestra.run.trial031.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2026-02-17T21:14:18.237 INFO:teuthology.orchestra.run.trial031.stdout:Recommended packages: 2026-02-17T21:14:18.237 INFO:teuthology.orchestra.run.trial031.stdout: btrfs-tools 2026-02-17T21:14:18.268 INFO:teuthology.orchestra.run.trial031.stdout:The following NEW packages will be installed: 2026-02-17T21:14:18.268 INFO:teuthology.orchestra.run.trial031.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-17T21:14:18.269 INFO:teuthology.orchestra.run.trial031.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-17T21:14:18.269 INFO:teuthology.orchestra.run.trial031.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2026-02-17T21:14:18.269 INFO:teuthology.orchestra.run.trial031.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 2026-02-17T21:14:18.269 INFO:teuthology.orchestra.run.trial031.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2026-02-17T21:14:18.269 INFO:teuthology.orchestra.run.trial031.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2026-02-17T21:14:18.269 INFO:teuthology.orchestra.run.trial031.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2026-02-17T21:14:18.269 INFO:teuthology.orchestra.run.trial031.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2026-02-17T21:14:18.269 INFO:teuthology.orchestra.run.trial031.stdout: python-pastedeploy-tpl python3-asyncssh python3-cachetools 2026-02-17T21:14:18.269 INFO:teuthology.orchestra.run.trial031.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cheroot 2026-02-17T21:14:18.269 INFO:teuthology.orchestra.run.trial031.stdout: python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-17T21:14:18.269 INFO:teuthology.orchestra.run.trial031.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-17T21:14:18.270 INFO:teuthology.orchestra.run.trial031.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-logutils 2026-02-17T21:14:18.270 INFO:teuthology.orchestra.run.trial031.stdout: python3-mako python3-natsort python3-paste python3-pastedeploy 2026-02-17T21:14:18.270 INFO:teuthology.orchestra.run.trial031.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2026-02-17T21:14:18.270 INFO:teuthology.orchestra.run.trial031.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-17T21:14:18.270 INFO:teuthology.orchestra.run.trial031.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2026-02-17T21:14:18.270 INFO:teuthology.orchestra.run.trial031.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2026-02-17T21:14:18.270 INFO:teuthology.orchestra.run.trial031.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2026-02-17T21:14:18.270 INFO:teuthology.orchestra.run.trial031.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2026-02-17T21:14:18.270 INFO:teuthology.orchestra.run.trial031.stdout: python3-threadpoolctl python3-toml python3-waitress python3-wcwidth 2026-02-17T21:14:18.270 INFO:teuthology.orchestra.run.trial031.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2026-02-17T21:14:18.270 INFO:teuthology.orchestra.run.trial031.stdout: python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse socat unzip 2026-02-17T21:14:18.271 INFO:teuthology.orchestra.run.trial031.stdout: xmlstarlet zip 2026-02-17T21:14:18.346 INFO:teuthology.orchestra.run.trial031.stdout:0 upgraded, 108 newly installed, 0 to remove and 53 not upgraded. 2026-02-17T21:14:18.346 INFO:teuthology.orchestra.run.trial031.stdout:Need to get 178 MB of archives. 2026-02-17T21:14:18.346 INFO:teuthology.orchestra.run.trial031.stdout:After this operation, 809 MB of additional disk space will be used. 2026-02-17T21:14:18.347 INFO:teuthology.orchestra.run.trial031.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.2.3-679-gd855f53b-1jammy [3598 kB] 2026-02-17T21:14:18.372 INFO:teuthology.orchestra.run.trial031.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-17T21:14:18.418 INFO:teuthology.orchestra.run.trial031.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-17T21:14:18.419 INFO:teuthology.orchestra.run.trial031.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-17T21:14:18.428 INFO:teuthology.orchestra.run.trial031.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-17T21:14:18.447 INFO:teuthology.orchestra.run.trial031.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.2.3-679-gd855f53b-1jammy [3257 kB] 2026-02-17T21:14:18.457 INFO:teuthology.orchestra.run.trial031.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-17T21:14:18.458 INFO:teuthology.orchestra.run.trial031.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-17T21:14:18.464 INFO:teuthology.orchestra.run.trial031.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-17T21:14:18.466 INFO:teuthology.orchestra.run.trial031.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-17T21:14:18.466 INFO:teuthology.orchestra.run.trial031.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-17T21:14:18.467 INFO:teuthology.orchestra.run.trial031.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-17T21:14:18.467 INFO:teuthology.orchestra.run.trial031.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-17T21:14:18.472 INFO:teuthology.orchestra.run.trial031.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2026-02-17T21:14:18.473 INFO:teuthology.orchestra.run.trial031.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2026-02-17T21:14:18.474 INFO:teuthology.orchestra.run.trial031.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2026-02-17T21:14:18.477 INFO:teuthology.orchestra.run.trial031.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5034 B] 2026-02-17T21:14:18.478 INFO:teuthology.orchestra.run.trial031.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2026-02-17T21:14:18.479 INFO:teuthology.orchestra.run.trial031.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2026-02-17T21:14:18.480 INFO:teuthology.orchestra.run.trial031.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2026-02-17T21:14:18.489 INFO:teuthology.orchestra.run.trial031.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-17T21:14:18.489 INFO:teuthology.orchestra.run.trial031.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-17T21:14:18.489 INFO:teuthology.orchestra.run.trial031.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-17T21:14:18.490 INFO:teuthology.orchestra.run.trial031.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-17T21:14:18.490 INFO:teuthology.orchestra.run.trial031.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-17T21:14:18.491 INFO:teuthology.orchestra.run.trial031.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-17T21:14:18.491 INFO:teuthology.orchestra.run.trial031.stdout:Get:27 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.2.3-679-gd855f53b-1jammy [979 kB] 2026-02-17T21:14:18.496 INFO:teuthology.orchestra.run.trial031.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-17T21:14:18.497 INFO:teuthology.orchestra.run.trial031.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-17T21:14:18.497 INFO:teuthology.orchestra.run.trial031.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-17T21:14:18.506 INFO:teuthology.orchestra.run.trial031.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-17T21:14:18.507 INFO:teuthology.orchestra.run.trial031.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-17T21:14:18.507 INFO:teuthology.orchestra.run.trial031.stdout:Get:33 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.2.3-679-gd855f53b-1jammy [357 kB] 2026-02-17T21:14:18.508 INFO:teuthology.orchestra.run.trial031.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2026-02-17T21:14:18.510 INFO:teuthology.orchestra.run.trial031.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2026-02-17T21:14:18.511 INFO:teuthology.orchestra.run.trial031.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2026-02-17T21:14:18.511 INFO:teuthology.orchestra.run.trial031.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7320 B] 2026-02-17T21:14:18.511 INFO:teuthology.orchestra.run.trial031.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-17T21:14:18.513 INFO:teuthology.orchestra.run.trial031.stdout:Get:39 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.2.3-679-gd855f53b-1jammy [32.9 kB] 2026-02-17T21:14:18.513 INFO:teuthology.orchestra.run.trial031.stdout:Get:40 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.2.3-679-gd855f53b-1jammy [184 kB] 2026-02-17T21:14:18.515 INFO:teuthology.orchestra.run.trial031.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-waitress all 1.4.4-1.1ubuntu1.1 [47.0 kB] 2026-02-17T21:14:18.516 INFO:teuthology.orchestra.run.trial031.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2026-02-17T21:14:18.516 INFO:teuthology.orchestra.run.trial031.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2026-02-17T21:14:18.516 INFO:teuthology.orchestra.run.trial031.stdout:Get:44 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.2.3-679-gd855f53b-1jammy [70.1 kB] 2026-02-17T21:14:18.518 INFO:teuthology.orchestra.run.trial031.stdout:Get:45 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.2.3-679-gd855f53b-1jammy [336 kB] 2026-02-17T21:14:18.523 INFO:teuthology.orchestra.run.trial031.stdout:Get:46 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.2.3-679-gd855f53b-1jammy [6934 kB] 2026-02-17T21:14:18.524 INFO:teuthology.orchestra.run.trial031.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4892 B] 2026-02-17T21:14:18.525 INFO:teuthology.orchestra.run.trial031.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2026-02-17T21:14:18.525 INFO:teuthology.orchestra.run.trial031.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2026-02-17T21:14:18.526 INFO:teuthology.orchestra.run.trial031.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2026-02-17T21:14:18.526 INFO:teuthology.orchestra.run.trial031.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.3 [181 kB] 2026-02-17T21:14:18.528 INFO:teuthology.orchestra.run.trial031.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-17T21:14:18.529 INFO:teuthology.orchestra.run.trial031.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-17T21:14:18.534 INFO:teuthology.orchestra.run.trial031.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-17T21:14:18.534 INFO:teuthology.orchestra.run.trial031.stdout:Get:55 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-17T21:14:18.535 INFO:teuthology.orchestra.run.trial031.stdout:Get:56 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-17T21:14:18.549 INFO:teuthology.orchestra.run.trial031.stdout:Get:57 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-17T21:14:18.551 INFO:teuthology.orchestra.run.trial031.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-17T21:14:18.551 INFO:teuthology.orchestra.run.trial031.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-17T21:14:18.559 INFO:teuthology.orchestra.run.trial031.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-17T21:14:18.560 INFO:teuthology.orchestra.run.trial031.stdout:Get:61 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-17T21:14:18.560 INFO:teuthology.orchestra.run.trial031.stdout:Get:62 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-17T21:14:18.560 INFO:teuthology.orchestra.run.trial031.stdout:Get:63 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-17T21:14:18.561 INFO:teuthology.orchestra.run.trial031.stdout:Get:64 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-17T21:14:18.561 INFO:teuthology.orchestra.run.trial031.stdout:Get:65 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-17T21:14:18.562 INFO:teuthology.orchestra.run.trial031.stdout:Get:66 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-17T21:14:18.563 INFO:teuthology.orchestra.run.trial031.stdout:Get:67 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-17T21:14:18.564 INFO:teuthology.orchestra.run.trial031.stdout:Get:68 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-17T21:14:18.572 INFO:teuthology.orchestra.run.trial031.stdout:Get:69 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-17T21:14:18.574 INFO:teuthology.orchestra.run.trial031.stdout:Get:70 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-17T21:14:18.575 INFO:teuthology.orchestra.run.trial031.stdout:Get:71 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2026-02-17T21:14:18.575 INFO:teuthology.orchestra.run.trial031.stdout:Get:72 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2026-02-17T21:14:18.576 INFO:teuthology.orchestra.run.trial031.stdout:Get:73 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-17T21:14:18.578 INFO:teuthology.orchestra.run.trial031.stdout:Get:74 https://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2026-02-17T21:14:18.578 INFO:teuthology.orchestra.run.trial031.stdout:Get:75 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-17T21:14:18.581 INFO:teuthology.orchestra.run.trial031.stdout:Get:76 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-17T21:14:18.582 INFO:teuthology.orchestra.run.trial031.stdout:Get:77 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2026-02-17T21:14:18.582 INFO:teuthology.orchestra.run.trial031.stdout:Get:78 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-17T21:14:18.591 INFO:teuthology.orchestra.run.trial031.stdout:Get:79 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-17T21:14:18.592 INFO:teuthology.orchestra.run.trial031.stdout:Get:80 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-17T21:14:18.592 INFO:teuthology.orchestra.run.trial031.stdout:Get:81 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-17T21:14:18.595 INFO:teuthology.orchestra.run.trial031.stdout:Get:82 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2026-02-17T21:14:18.596 INFO:teuthology.orchestra.run.trial031.stdout:Get:83 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-17T21:14:18.596 INFO:teuthology.orchestra.run.trial031.stdout:Get:84 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-17T21:14:18.597 INFO:teuthology.orchestra.run.trial031.stdout:Get:85 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-17T21:14:18.600 INFO:teuthology.orchestra.run.trial031.stdout:Get:86 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-17T21:14:18.637 INFO:teuthology.orchestra.run.trial031.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.2.3-679-gd855f53b-1jammy [112 kB] 2026-02-17T21:14:18.638 INFO:teuthology.orchestra.run.trial031.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.2.3-679-gd855f53b-1jammy [470 kB] 2026-02-17T21:14:18.645 INFO:teuthology.orchestra.run.trial031.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.2.3-679-gd855f53b-1jammy [26.5 MB] 2026-02-17T21:14:19.429 INFO:teuthology.orchestra.run.trial031.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.2.3-679-gd855f53b-1jammy [5178 kB] 2026-02-17T21:14:19.509 INFO:teuthology.orchestra.run.trial031.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.2.3-679-gd855f53b-1jammy [248 kB] 2026-02-17T21:14:19.510 INFO:teuthology.orchestra.run.trial031.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.2.3-679-gd855f53b-1jammy [125 kB] 2026-02-17T21:14:19.511 INFO:teuthology.orchestra.run.trial031.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.2.3-679-gd855f53b-1jammy [1082 kB] 2026-02-17T21:14:19.530 INFO:teuthology.orchestra.run.trial031.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.2.3-679-gd855f53b-1jammy [6239 kB] 2026-02-17T21:14:19.644 INFO:teuthology.orchestra.run.trial031.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.2.3-679-gd855f53b-1jammy [23.0 MB] 2026-02-17T21:14:20.609 INFO:teuthology.orchestra.run.trial031.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.2.3-679-gd855f53b-1jammy [14.2 kB] 2026-02-17T21:14:20.610 INFO:teuthology.orchestra.run.trial031.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.2.3-679-gd855f53b-1jammy [1173 kB] 2026-02-17T21:14:20.624 INFO:teuthology.orchestra.run.trial031.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.2.3-679-gd855f53b-1jammy [2503 kB] 2026-02-17T21:14:20.666 INFO:teuthology.orchestra.run.trial031.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.2.3-679-gd855f53b-1jammy [798 kB] 2026-02-17T21:14:20.678 INFO:teuthology.orchestra.run.trial031.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.2.3-679-gd855f53b-1jammy [157 kB] 2026-02-17T21:14:20.681 INFO:teuthology.orchestra.run.trial031.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.2.3-679-gd855f53b-1jammy [2396 kB] 2026-02-17T21:14:20.720 INFO:teuthology.orchestra.run.trial031.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.2.3-679-gd855f53b-1jammy [8625 kB] 2026-02-17T21:14:20.863 INFO:teuthology.orchestra.run.trial031.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.2.3-679-gd855f53b-1jammy [14.3 kB] 2026-02-17T21:14:20.864 INFO:teuthology.orchestra.run.trial031.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.2.3-679-gd855f53b-1jammy [52.1 MB] 2026-02-17T21:14:21.724 INFO:teuthology.orchestra.run.trial031.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.2.3-679-gd855f53b-1jammy [135 kB] 2026-02-17T21:14:21.725 INFO:teuthology.orchestra.run.trial031.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.2.3-679-gd855f53b-1jammy [41.0 kB] 2026-02-17T21:14:21.725 INFO:teuthology.orchestra.run.trial031.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.2.3-679-gd855f53b-1jammy [13.7 MB] 2026-02-17T21:14:21.945 INFO:teuthology.orchestra.run.trial031.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-17-1732-squid/d855f53b89fdcec760fd9232a5fb55ed4fb111a1/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.2.3-679-gd855f53b-1jammy [92.1 kB] 2026-02-17T21:14:22.271 INFO:teuthology.orchestra.run.trial031.stdout:Fetched 178 MB in 4s (48.3 MB/s) 2026-02-17T21:14:22.287 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-17T21:14:22.442 INFO:teuthology.orchestra.run.trial031.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167438 files and directories currently installed.) 2026-02-17T21:14:22.444 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-17T21:14:22.445 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-17T21:14:22.465 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-17T21:14:22.476 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-17T21:14:22.476 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-17T21:14:22.492 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-17T21:14:22.504 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-17T21:14:22.504 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-17T21:14:22.522 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-17T21:14:22.533 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-17T21:14:22.538 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T21:14:22.569 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-17T21:14:22.577 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-17T21:14:22.578 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T21:14:22.594 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-17T21:14:22.602 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-17T21:14:22.602 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T21:14:22.622 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-17T21:14:22.630 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-17T21:14:22.630 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-17T21:14:22.647 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package librados2. 2026-02-17T21:14:22.655 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../007-librados2_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:22.655 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking librados2 (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:22.696 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package libnbd0. 2026-02-17T21:14:22.705 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2026-02-17T21:14:22.705 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-17T21:14:22.719 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package librbd1. 2026-02-17T21:14:22.727 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../009-librbd1_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:22.727 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking librbd1 (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:22.771 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package libcephfs2. 2026-02-17T21:14:22.779 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../010-libcephfs2_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:22.779 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking libcephfs2 (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:22.800 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-rados. 2026-02-17T21:14:22.807 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../011-python3-rados_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:22.808 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-rados (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:22.825 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-17T21:14:22.833 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.2.3-679-gd855f53b-1jammy_all.deb ... 2026-02-17T21:14:22.833 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-ceph-argparse (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:22.847 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-cephfs. 2026-02-17T21:14:22.854 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../013-python3-cephfs_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:22.855 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-cephfs (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:22.870 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-17T21:14:22.879 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../014-python3-ceph-common_19.2.3-679-gd855f53b-1jammy_all.deb ... 2026-02-17T21:14:22.879 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-ceph-common (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:22.896 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-17T21:14:22.903 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-17T21:14:22.904 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-17T21:14:22.919 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-prettytable. 2026-02-17T21:14:22.927 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2026-02-17T21:14:22.927 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-17T21:14:22.942 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-rbd. 2026-02-17T21:14:22.950 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../017-python3-rbd_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:22.950 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-rbd (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:22.966 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-17T21:14:22.975 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-17T21:14:22.975 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-17T21:14:22.993 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package libreadline-dev:amd64. 2026-02-17T21:14:23.001 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2026-02-17T21:14:23.001 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2026-02-17T21:14:23.019 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2026-02-17T21:14:23.027 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2026-02-17T21:14:23.027 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2026-02-17T21:14:23.045 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package lua5.1. 2026-02-17T21:14:23.053 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2026-02-17T21:14:23.053 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2026-02-17T21:14:23.073 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package lua-any. 2026-02-17T21:14:23.081 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2026-02-17T21:14:23.081 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking lua-any (27ubuntu1) ... 2026-02-17T21:14:23.095 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package zip. 2026-02-17T21:14:23.103 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2026-02-17T21:14:23.103 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking zip (3.0-12build2) ... 2026-02-17T21:14:23.118 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package unzip. 2026-02-17T21:14:23.126 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2026-02-17T21:14:23.127 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2026-02-17T21:14:23.143 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package luarocks. 2026-02-17T21:14:23.151 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2026-02-17T21:14:23.151 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2026-02-17T21:14:23.189 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package librgw2. 2026-02-17T21:14:23.197 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../026-librgw2_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:23.198 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking librgw2 (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:23.270 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-rgw. 2026-02-17T21:14:23.278 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../027-python3-rgw_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:23.278 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-rgw (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:23.293 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-17T21:14:23.301 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../028-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-17T21:14:23.302 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-17T21:14:23.317 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package libradosstriper1. 2026-02-17T21:14:23.325 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../029-libradosstriper1_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:23.325 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking libradosstriper1 (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:23.343 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package ceph-common. 2026-02-17T21:14:23.351 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../030-ceph-common_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:23.351 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking ceph-common (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:23.552 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package ceph-base. 2026-02-17T21:14:23.560 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../031-ceph-base_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:23.564 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking ceph-base (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:23.629 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-17T21:14:23.637 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-17T21:14:23.638 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-17T21:14:23.651 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-cheroot. 2026-02-17T21:14:23.659 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-17T21:14:23.659 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-17T21:14:23.676 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-17T21:14:23.684 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-17T21:14:23.685 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-17T21:14:23.699 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-17T21:14:23.707 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-17T21:14:23.708 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-17T21:14:23.721 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-17T21:14:23.730 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-17T21:14:23.730 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-17T21:14:23.745 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-tempora. 2026-02-17T21:14:23.753 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2026-02-17T21:14:23.754 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-17T21:14:23.768 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-portend. 2026-02-17T21:14:23.776 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2026-02-17T21:14:23.776 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-17T21:14:23.790 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-17T21:14:23.798 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-17T21:14:23.798 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-17T21:14:23.813 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-17T21:14:23.821 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-17T21:14:23.821 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-17T21:14:23.844 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-natsort. 2026-02-17T21:14:23.852 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2026-02-17T21:14:23.853 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-17T21:14:23.869 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-logutils. 2026-02-17T21:14:23.877 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2026-02-17T21:14:23.878 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-logutils (0.3.3-8) ... 2026-02-17T21:14:23.892 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-mako. 2026-02-17T21:14:23.900 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2026-02-17T21:14:23.900 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2026-02-17T21:14:23.917 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-simplegeneric. 2026-02-17T21:14:23.925 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2026-02-17T21:14:23.926 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2026-02-17T21:14:23.939 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-singledispatch. 2026-02-17T21:14:23.947 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2026-02-17T21:14:23.947 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2026-02-17T21:14:23.961 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-webob. 2026-02-17T21:14:23.969 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-17T21:14:23.969 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-17T21:14:23.986 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-waitress. 2026-02-17T21:14:23.994 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1.1_all.deb ... 2026-02-17T21:14:23.996 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1.1) ... 2026-02-17T21:14:24.012 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-tempita. 2026-02-17T21:14:24.020 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2026-02-17T21:14:24.020 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2026-02-17T21:14:24.034 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-paste. 2026-02-17T21:14:24.042 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2026-02-17T21:14:24.043 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2026-02-17T21:14:24.071 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2026-02-17T21:14:24.080 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2026-02-17T21:14:24.080 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2026-02-17T21:14:24.093 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-pastedeploy. 2026-02-17T21:14:24.101 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2026-02-17T21:14:24.102 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2026-02-17T21:14:24.117 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-webtest. 2026-02-17T21:14:24.125 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2026-02-17T21:14:24.126 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-webtest (2.0.35-1) ... 2026-02-17T21:14:24.142 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-pecan. 2026-02-17T21:14:24.150 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2026-02-17T21:14:24.150 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2026-02-17T21:14:24.175 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-werkzeug. 2026-02-17T21:14:24.183 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.3_all.deb ... 2026-02-17T21:14:24.183 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.3) ... 2026-02-17T21:14:24.202 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-17T21:14:24.210 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_19.2.3-679-gd855f53b-1jammy_all.deb ... 2026-02-17T21:14:24.211 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking ceph-mgr-modules-core (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:24.239 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-17T21:14:24.247 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:24.247 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking libsqlite3-mod-ceph (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:24.262 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package ceph-mgr. 2026-02-17T21:14:24.270 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../057-ceph-mgr_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:24.271 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking ceph-mgr (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:24.293 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package ceph-mon. 2026-02-17T21:14:24.301 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../058-ceph-mon_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:24.301 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking ceph-mon (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:24.356 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-17T21:14:24.365 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../059-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-17T21:14:24.365 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-17T21:14:24.382 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package ceph-osd. 2026-02-17T21:14:24.390 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../060-ceph-osd_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:24.390 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking ceph-osd (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:24.552 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package ceph. 2026-02-17T21:14:24.560 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../061-ceph_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:24.561 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking ceph (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:24.575 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package ceph-fuse. 2026-02-17T21:14:24.583 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../062-ceph-fuse_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:24.583 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking ceph-fuse (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:24.606 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package ceph-mds. 2026-02-17T21:14:24.614 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../063-ceph-mds_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:24.615 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking ceph-mds (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:24.648 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package cephadm. 2026-02-17T21:14:24.657 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../064-cephadm_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:24.657 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking cephadm (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:24.672 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-17T21:14:24.681 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-17T21:14:24.681 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-17T21:14:24.702 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-17T21:14:24.710 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../066-ceph-mgr-cephadm_19.2.3-679-gd855f53b-1jammy_all.deb ... 2026-02-17T21:14:24.711 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking ceph-mgr-cephadm (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:24.730 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-17T21:14:24.739 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../067-python3-repoze.lru_0.7-2_all.deb ... 2026-02-17T21:14:24.739 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-17T21:14:24.754 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-routes. 2026-02-17T21:14:24.762 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../068-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-17T21:14:24.762 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-17T21:14:24.782 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-17T21:14:24.790 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../069-ceph-mgr-dashboard_19.2.3-679-gd855f53b-1jammy_all.deb ... 2026-02-17T21:14:24.790 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking ceph-mgr-dashboard (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:24.990 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-17T21:14:24.999 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-17T21:14:24.999 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-17T21:14:25.038 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-joblib. 2026-02-17T21:14:25.047 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../071-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-17T21:14:25.047 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-17T21:14:25.072 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-17T21:14:25.081 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../072-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-17T21:14:25.081 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-17T21:14:25.096 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-sklearn. 2026-02-17T21:14:25.104 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../073-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-17T21:14:25.104 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-17T21:14:25.183 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-17T21:14:25.192 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../074-ceph-mgr-diskprediction-local_19.2.3-679-gd855f53b-1jammy_all.deb ... 2026-02-17T21:14:25.192 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking ceph-mgr-diskprediction-local (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:25.362 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-cachetools. 2026-02-17T21:14:25.370 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../075-python3-cachetools_5.0.0-1_all.deb ... 2026-02-17T21:14:25.370 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-17T21:14:25.385 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-rsa. 2026-02-17T21:14:25.393 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../076-python3-rsa_4.8-1_all.deb ... 2026-02-17T21:14:25.394 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-17T21:14:25.410 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-google-auth. 2026-02-17T21:14:25.419 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-3_all.deb ... 2026-02-17T21:14:25.419 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-17T21:14:25.436 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-17T21:14:25.444 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-17T21:14:25.445 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-17T21:14:25.461 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-websocket. 2026-02-17T21:14:25.469 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../079-python3-websocket_1.2.3-1_all.deb ... 2026-02-17T21:14:25.469 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-17T21:14:25.487 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-17T21:14:25.495 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../080-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-17T21:14:25.495 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-17T21:14:25.591 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-17T21:14:25.599 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_19.2.3-679-gd855f53b-1jammy_all.deb ... 2026-02-17T21:14:25.600 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking ceph-mgr-k8sevents (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:25.614 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-17T21:14:25.622 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../082-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-17T21:14:25.629 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-17T21:14:25.645 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-17T21:14:25.654 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../083-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-17T21:14:25.654 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-17T21:14:25.668 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package jq. 2026-02-17T21:14:25.677 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../084-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-17T21:14:25.678 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-17T21:14:25.692 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package socat. 2026-02-17T21:14:25.700 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../085-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-17T21:14:25.700 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-17T21:14:25.720 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package xmlstarlet. 2026-02-17T21:14:25.728 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-17T21:14:25.729 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-17T21:14:25.765 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package ceph-test. 2026-02-17T21:14:25.773 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../087-ceph-test_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:25.774 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking ceph-test (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:26.235 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package ceph-volume. 2026-02-17T21:14:26.244 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../088-ceph-volume_19.2.3-679-gd855f53b-1jammy_all.deb ... 2026-02-17T21:14:26.244 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking ceph-volume (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:26.268 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-17T21:14:26.276 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../089-libcephfs-dev_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:26.276 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking libcephfs-dev (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:26.291 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package lua-socket:amd64. 2026-02-17T21:14:26.299 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../090-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2026-02-17T21:14:26.300 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2026-02-17T21:14:26.320 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package lua-sec:amd64. 2026-02-17T21:14:26.328 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../091-lua-sec_1.0.2-1_amd64.deb ... 2026-02-17T21:14:26.328 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2026-02-17T21:14:26.345 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package nvme-cli. 2026-02-17T21:14:26.353 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../092-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-17T21:14:26.354 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-17T21:14:26.387 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package pkg-config. 2026-02-17T21:14:26.395 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../093-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2026-02-17T21:14:26.396 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2026-02-17T21:14:26.411 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-17T21:14:26.419 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../094-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-17T21:14:26.419 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-17T21:14:26.448 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-17T21:14:26.456 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../095-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-17T21:14:26.457 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-17T21:14:26.471 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-pastescript. 2026-02-17T21:14:26.479 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../096-python3-pastescript_2.0.2-4_all.deb ... 2026-02-17T21:14:26.479 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2026-02-17T21:14:26.498 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-pluggy. 2026-02-17T21:14:26.506 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../097-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-17T21:14:26.506 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-17T21:14:26.522 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-psutil. 2026-02-17T21:14:26.530 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../098-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-17T21:14:26.531 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-17T21:14:26.549 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-py. 2026-02-17T21:14:26.557 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../099-python3-py_1.10.0-1_all.deb ... 2026-02-17T21:14:26.557 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-17T21:14:26.578 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-pygments. 2026-02-17T21:14:26.586 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../100-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-17T21:14:26.587 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-17T21:14:26.628 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-pyinotify. 2026-02-17T21:14:26.637 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../101-python3-pyinotify_0.9.6-1.3_all.deb ... 2026-02-17T21:14:26.637 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2026-02-17T21:14:26.652 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-toml. 2026-02-17T21:14:26.660 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../102-python3-toml_0.10.2-1_all.deb ... 2026-02-17T21:14:26.660 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-17T21:14:26.675 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-pytest. 2026-02-17T21:14:26.683 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../103-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-17T21:14:26.684 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-17T21:14:26.705 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package python3-simplejson. 2026-02-17T21:14:26.714 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../104-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-17T21:14:26.714 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-17T21:14:26.731 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-17T21:14:26.740 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../105-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-17T21:14:26.740 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-17T21:14:26.806 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package radosgw. 2026-02-17T21:14:26.815 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../106-radosgw_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:26.815 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking radosgw (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:26.930 INFO:teuthology.orchestra.run.trial031.stdout:Selecting previously unselected package rbd-fuse. 2026-02-17T21:14:26.938 INFO:teuthology.orchestra.run.trial031.stdout:Preparing to unpack .../107-rbd-fuse_19.2.3-679-gd855f53b-1jammy_amd64.deb ... 2026-02-17T21:14:26.939 INFO:teuthology.orchestra.run.trial031.stdout:Unpacking rbd-fuse (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:26.967 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-17T21:14:27.037 INFO:teuthology.orchestra.run.trial031.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-17T21:14:27.037 INFO:teuthology.orchestra.run.trial031.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-17T21:14:27.111 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-17T21:14:27.401 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-17T21:14:27.790 INFO:teuthology.orchestra.run.trial031.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-17T21:14:27.801 INFO:teuthology.orchestra.run.trial031.stdout:Setting up cephadm (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:27.843 INFO:teuthology.orchestra.run.trial031.stdout:Adding system user cephadm....done 2026-02-17T21:14:27.854 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1.1) ... 2026-02-17T21:14:27.925 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-17T21:14:27.997 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-17T21:14:27.998 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-17T21:14:28.074 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-17T21:14:28.150 INFO:teuthology.orchestra.run.trial031.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-17T21:14:28.150 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-17T21:14:28.247 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-17T21:14:28.370 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-17T21:14:28.444 INFO:teuthology.orchestra.run.trial031.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2026-02-17T21:14:28.454 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2026-02-17T21:14:28.533 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-17T21:14:28.608 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-ceph-argparse (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:28.676 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-17T21:14:28.677 INFO:teuthology.orchestra.run.trial031.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-17T21:14:28.678 INFO:teuthology.orchestra.run.trial031.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2026-02-17T21:14:28.679 INFO:teuthology.orchestra.run.trial031.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2026-02-17T21:14:28.680 INFO:teuthology.orchestra.run.trial031.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-17T21:14:28.681 INFO:teuthology.orchestra.run.trial031.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2026-02-17T21:14:28.697 INFO:teuthology.orchestra.run.trial031.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2026-02-17T21:14:28.699 INFO:teuthology.orchestra.run.trial031.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2026-02-17T21:14:28.700 INFO:teuthology.orchestra.run.trial031.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-17T21:14:28.701 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-17T21:14:28.826 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-17T21:14:28.904 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-17T21:14:28.984 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-17T21:14:29.075 INFO:teuthology.orchestra.run.trial031.stdout:Setting up zip (3.0-12build2) ... 2026-02-17T21:14:29.076 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-17T21:14:29.336 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2026-02-17T21:14:29.411 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2026-02-17T21:14:29.411 INFO:teuthology.orchestra.run.trial031.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-17T21:14:29.412 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-17T21:14:29.509 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-17T21:14:29.635 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2026-02-17T21:14:29.764 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-17T21:14:29.847 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.3) ... 2026-02-17T21:14:29.953 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-17T21:14:30.027 INFO:teuthology.orchestra.run.trial031.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-17T21:14:30.028 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-ceph-common (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:30.110 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-17T21:14:30.625 INFO:teuthology.orchestra.run.trial031.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2026-02-17T21:14:30.647 INFO:teuthology.orchestra.run.trial031.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T21:14:30.652 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-17T21:14:30.729 INFO:teuthology.orchestra.run.trial031.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-17T21:14:30.730 INFO:teuthology.orchestra.run.trial031.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-17T21:14:30.731 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-17T21:14:30.805 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-17T21:14:30.881 INFO:teuthology.orchestra.run.trial031.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T21:14:30.882 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-17T21:14:30.960 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2026-02-17T21:14:31.034 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-logutils (0.3.3-8) ... 2026-02-17T21:14:31.110 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-17T21:14:31.184 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2026-02-17T21:14:31.261 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-17T21:14:31.340 INFO:teuthology.orchestra.run.trial031.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-17T21:14:31.341 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-17T21:14:31.424 INFO:teuthology.orchestra.run.trial031.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-17T21:14:31.425 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-17T21:14:31.502 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2026-02-17T21:14:31.581 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-17T21:14:31.667 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-17T21:14:31.743 INFO:teuthology.orchestra.run.trial031.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2026-02-17T21:14:31.744 INFO:teuthology.orchestra.run.trial031.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2026-02-17T21:14:31.745 INFO:teuthology.orchestra.run.trial031.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-17T21:14:31.746 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-17T21:14:31.885 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2026-02-17T21:14:31.965 INFO:teuthology.orchestra.run.trial031.stdout:Setting up lua-any (27ubuntu1) ... 2026-02-17T21:14:31.966 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-17T21:14:32.039 INFO:teuthology.orchestra.run.trial031.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T21:14:32.040 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-17T21:14:32.123 INFO:teuthology.orchestra.run.trial031.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-17T21:14:32.124 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-webtest (2.0.35-1) ... 2026-02-17T21:14:32.205 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-17T21:14:32.344 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-pastescript (2.0.2-4) ... 2026-02-17T21:14:32.434 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2026-02-17T21:14:32.550 INFO:teuthology.orchestra.run.trial031.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-17T21:14:32.551 INFO:teuthology.orchestra.run.trial031.stdout:Setting up librados2 (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:32.552 INFO:teuthology.orchestra.run.trial031.stdout:Setting up libsqlite3-mod-ceph (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:32.553 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-17T21:14:33.087 INFO:teuthology.orchestra.run.trial031.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2026-02-17T21:14:33.098 INFO:teuthology.orchestra.run.trial031.stdout:Setting up libcephfs2 (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:33.099 INFO:teuthology.orchestra.run.trial031.stdout:Setting up libradosstriper1 (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:33.100 INFO:teuthology.orchestra.run.trial031.stdout:Setting up librbd1 (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:33.100 INFO:teuthology.orchestra.run.trial031.stdout:Setting up ceph-mgr-modules-core (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:33.101 INFO:teuthology.orchestra.run.trial031.stdout:Setting up ceph-fuse (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:33.174 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-17T21:14:33.174 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-17T21:14:33.606 INFO:teuthology.orchestra.run.trial031.stdout:Setting up libcephfs-dev (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:33.607 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-rados (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:33.608 INFO:teuthology.orchestra.run.trial031.stdout:Setting up librgw2 (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:33.609 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-rbd (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:33.610 INFO:teuthology.orchestra.run.trial031.stdout:Setting up rbd-fuse (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:33.611 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-rgw (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:33.612 INFO:teuthology.orchestra.run.trial031.stdout:Setting up python3-cephfs (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:33.613 INFO:teuthology.orchestra.run.trial031.stdout:Setting up ceph-common (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:33.641 INFO:teuthology.orchestra.run.trial031.stdout:Adding group ceph....done 2026-02-17T21:14:33.677 INFO:teuthology.orchestra.run.trial031.stdout:Adding system user ceph....done 2026-02-17T21:14:33.687 INFO:teuthology.orchestra.run.trial031.stdout:Setting system user ceph properties....done 2026-02-17T21:14:33.692 INFO:teuthology.orchestra.run.trial031.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-17T21:14:33.771 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-17T21:14:34.051 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-17T21:14:34.442 INFO:teuthology.orchestra.run.trial031.stdout:Setting up ceph-test (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:34.442 INFO:teuthology.orchestra.run.trial031.stdout:Setting up radosgw (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:34.708 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-17T21:14:34.709 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-17T21:14:35.102 INFO:teuthology.orchestra.run.trial031.stdout:Setting up ceph-base (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:35.219 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-17T21:14:35.624 INFO:teuthology.orchestra.run.trial031.stdout:Setting up ceph-mds (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:35.684 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-17T21:14:35.685 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-17T21:14:36.063 INFO:teuthology.orchestra.run.trial031.stdout:Setting up ceph-mgr (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:36.140 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-17T21:14:36.140 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-17T21:14:36.522 INFO:teuthology.orchestra.run.trial031.stdout:Setting up ceph-osd (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:36.609 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-17T21:14:36.610 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-17T21:14:36.995 INFO:teuthology.orchestra.run.trial031.stdout:Setting up ceph-mgr-k8sevents (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:36.995 INFO:teuthology.orchestra.run.trial031.stdout:Setting up ceph-mgr-diskprediction-local (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:37.007 INFO:teuthology.orchestra.run.trial031.stdout:Setting up ceph-mon (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:37.083 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-17T21:14:37.083 INFO:teuthology.orchestra.run.trial031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-17T21:14:37.483 INFO:teuthology.orchestra.run.trial031.stdout:Setting up ceph-mgr-cephadm (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:37.493 INFO:teuthology.orchestra.run.trial031.stdout:Setting up ceph (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:37.494 INFO:teuthology.orchestra.run.trial031.stdout:Setting up ceph-mgr-dashboard (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:37.505 INFO:teuthology.orchestra.run.trial031.stdout:Setting up ceph-volume (19.2.3-679-gd855f53b-1jammy) ... 2026-02-17T21:14:37.616 INFO:teuthology.orchestra.run.trial031.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2026-02-17T21:14:37.624 INFO:teuthology.orchestra.run.trial031.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.11) ... 2026-02-17T21:14:37.642 INFO:teuthology.orchestra.run.trial031.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-17T21:14:37.731 INFO:teuthology.orchestra.run.trial031.stdout:Processing triggers for install-info (6.8-4build1) ... 2026-02-17T21:14:38.215 INFO:teuthology.orchestra.run.trial031.stdout: 2026-02-17T21:14:38.215 INFO:teuthology.orchestra.run.trial031.stdout:Running kernel seems to be up-to-date. 2026-02-17T21:14:38.215 INFO:teuthology.orchestra.run.trial031.stdout: 2026-02-17T21:14:38.215 INFO:teuthology.orchestra.run.trial031.stdout:The processor microcode seems to be up-to-date. 2026-02-17T21:14:38.215 INFO:teuthology.orchestra.run.trial031.stdout: 2026-02-17T21:14:38.216 INFO:teuthology.orchestra.run.trial031.stdout:No services need to be restarted. 2026-02-17T21:14:38.216 INFO:teuthology.orchestra.run.trial031.stdout: 2026-02-17T21:14:38.216 INFO:teuthology.orchestra.run.trial031.stdout:No containers need to be restarted. 2026-02-17T21:14:38.216 INFO:teuthology.orchestra.run.trial031.stdout: 2026-02-17T21:14:38.216 INFO:teuthology.orchestra.run.trial031.stdout:No user sessions are running outdated binaries. 2026-02-17T21:14:38.216 INFO:teuthology.orchestra.run.trial031.stdout: 2026-02-17T21:14:38.216 INFO:teuthology.orchestra.run.trial031.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-17T21:14:38.863 INFO:teuthology.orchestra.run.trial031.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-17T21:14:38.869 DEBUG:teuthology.parallel:result is None 2026-02-17T21:14:38.870 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d855f53b89fdcec760fd9232a5fb55ed4fb111a1 2026-02-17T21:14:39.021 DEBUG:teuthology.orchestra.run.trial031:> dpkg-query -W -f '${Version}' ceph 2026-02-17T21:14:39.031 INFO:teuthology.orchestra.run.trial031.stdout:19.2.3-679-gd855f53b-1jammy 2026-02-17T21:14:39.032 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-679-gd855f53b-1jammy 2026-02-17T21:14:39.032 INFO:teuthology.task.install:The correct ceph version 19.2.3-679-gd855f53b-1jammy is installed. 2026-02-17T21:14:39.034 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-17T21:14:39.035 DEBUG:teuthology.orchestra.run.trial031:> set -ex 2026-02-17T21:14:39.035 DEBUG:teuthology.orchestra.run.trial031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-17T21:14:39.081 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-17T21:14:39.081 DEBUG:teuthology.orchestra.run.trial031:> set -ex 2026-02-17T21:14:39.081 DEBUG:teuthology.orchestra.run.trial031:> sudo dd of=/usr/bin/daemon-helper 2026-02-17T21:14:39.132 DEBUG:teuthology.orchestra.run.trial031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-17T21:14:39.180 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-17T21:14:39.180 DEBUG:teuthology.orchestra.run.trial031:> set -ex 2026-02-17T21:14:39.180 DEBUG:teuthology.orchestra.run.trial031:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-17T21:14:39.228 DEBUG:teuthology.orchestra.run.trial031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-17T21:14:39.276 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-17T21:14:39.276 DEBUG:teuthology.orchestra.run.trial031:> set -ex 2026-02-17T21:14:39.276 DEBUG:teuthology.orchestra.run.trial031:> sudo dd of=/usr/bin/stdin-killer 2026-02-17T21:14:39.324 DEBUG:teuthology.orchestra.run.trial031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-17T21:14:39.372 INFO:teuthology.run_tasks:Running task workunit... 2026-02-17T21:14:39.402 INFO:tasks.workunit:Pulling workunits from ref d855f53b89fdcec760fd9232a5fb55ed4fb111a1 2026-02-17T21:14:39.402 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-17T21:14:39.402 INFO:tasks.workunit:timeout=3h 2026-02-17T21:14:39.402 INFO:tasks.workunit:cleanup=True 2026-02-17T21:14:39.403 DEBUG:teuthology.orchestra.run.trial031:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-17T21:14:39.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T21:14:39.416 INFO:teuthology.orchestra.run.trial031.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-17T21:14:39.417 DEBUG:teuthology.orchestra.run.trial031:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-17T21:14:39.464 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-17T21:14:39.464 DEBUG:teuthology.orchestra.run.trial031:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-17T21:14:39.509 DEBUG:teuthology.orchestra.run.trial031:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d855f53b89fdcec760fd9232a5fb55ed4fb111a1 2026-02-17T21:14:39.563 INFO:tasks.workunit.client.0.trial031.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-17T21:15:01.157 INFO:tasks.workunit.client.0.trial031.stderr:Note: switching to 'd855f53b89fdcec760fd9232a5fb55ed4fb111a1'. 2026-02-17T21:15:01.158 INFO:tasks.workunit.client.0.trial031.stderr: 2026-02-17T21:15:01.158 INFO:tasks.workunit.client.0.trial031.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-17T21:15:01.158 INFO:tasks.workunit.client.0.trial031.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-17T21:15:01.158 INFO:tasks.workunit.client.0.trial031.stderr:state without impacting any branches by switching back to a branch. 2026-02-17T21:15:01.158 INFO:tasks.workunit.client.0.trial031.stderr: 2026-02-17T21:15:01.158 INFO:tasks.workunit.client.0.trial031.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-17T21:15:01.158 INFO:tasks.workunit.client.0.trial031.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-17T21:15:01.158 INFO:tasks.workunit.client.0.trial031.stderr: 2026-02-17T21:15:01.159 INFO:tasks.workunit.client.0.trial031.stderr: git switch -c 2026-02-17T21:15:01.159 INFO:tasks.workunit.client.0.trial031.stderr: 2026-02-17T21:15:01.159 INFO:tasks.workunit.client.0.trial031.stderr:Or undo this operation with: 2026-02-17T21:15:01.159 INFO:tasks.workunit.client.0.trial031.stderr: 2026-02-17T21:15:01.159 INFO:tasks.workunit.client.0.trial031.stderr: git switch - 2026-02-17T21:15:01.159 INFO:tasks.workunit.client.0.trial031.stderr: 2026-02-17T21:15:01.159 INFO:tasks.workunit.client.0.trial031.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-17T21:15:01.159 INFO:tasks.workunit.client.0.trial031.stderr: 2026-02-17T21:15:01.159 INFO:tasks.workunit.client.0.trial031.stderr:HEAD is now at d855f53b89f Merge branch 'wip-74104-squid' of https://github.com/cbodley/ceph into wip-yuri6-testing-2026-02-17-1732-squid 2026-02-17T21:15:01.168 DEBUG:teuthology.orchestra.run.trial031:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-17T21:15:01.214 DEBUG:teuthology.orchestra.run.trial031:> set -ex 2026-02-17T21:15:01.214 DEBUG:teuthology.orchestra.run.trial031:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-17T21:15:01.261 INFO:tasks.workunit:Running workunits matching osd on client.0... 2026-02-17T21:15:01.263 INFO:tasks.workunit:Running workunit osd/bad-inc-map.sh... 2026-02-17T21:15:01.263 DEBUG:teuthology.orchestra.run.trial031:workunit test osd/bad-inc-map.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d855f53b89fdcec760fd9232a5fb55ed4fb111a1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh 2026-02-17T21:15:01.314 INFO:tasks.workunit.client.0.trial031.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T21:15:01.331 INFO:tasks.workunit.client.0.trial031.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T21:15:01.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T21:15:01.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T21:15:01.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T21:15:01.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T21:15:01.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T21:15:01.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T21:15:01.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T21:15:01.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T21:15:01.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/bad-inc-map 2026-02-17T21:15:01.333 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:8: run: local dir=td/bad-inc-map 2026-02-17T21:15:01.333 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:9: run: shift 2026-02-17T21:15:01.333 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:11: run: export CEPH_MON=127.0.0.1:52089 2026-02-17T21:15:01.333 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:11: run: CEPH_MON=127.0.0.1:52089 2026-02-17T21:15:01.333 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:12: run: export CEPH_ARGS 2026-02-17T21:15:01.333 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:13: run: uuidgen 2026-02-17T21:15:01.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:13: run: CEPH_ARGS+='--fsid=af30db47-6aab-48c3-a6a9-80b65597d8b3 --auth-supported=none ' 2026-02-17T21:15:01.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:14: run: CEPH_ARGS+='--mon-host=127.0.0.1:52089 ' 2026-02-17T21:15:01.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:15: run: set -e 2026-02-17T21:15:01.334 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: set 2026-02-17T21:15:01.334 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T21:15:01.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: local funcs=TEST_bad_inc_map 2026-02-17T21:15:01.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:18: run: for func in $funcs 2026-02-17T21:15:01.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:19: run: setup td/bad-inc-map 2026-02-17T21:15:01.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/bad-inc-map 2026-02-17T21:15:01.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/bad-inc-map 2026-02-17T21:15:01.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/bad-inc-map 2026-02-17T21:15:01.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:15:01.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/bad-inc-map KILL 2026-02-17T21:15:01.337 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:15:01.337 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:15:01.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:15:01.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:15:01.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:15:01.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:15:01.338 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:15:01.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:15:01.339 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:15:01.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:15:01.340 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:15:01.340 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:15:01.340 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:15:01.341 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:15:01.341 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:15:01.341 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:15:01.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:15:01.343 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:15:01.343 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/bad-inc-map 2026-02-17T21:15:01.344 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:15:01.345 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:01.345 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.7918 2026-02-17T21:15:01.345 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.7918 2026-02-17T21:15:01.346 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:15:01.346 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:15:01.346 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/bad-inc-map 2026-02-17T21:15:01.347 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T21:15:01.347 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:01.347 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.7918 2026-02-17T21:15:01.347 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.7918 2026-02-17T21:15:01.348 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T21:15:01.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T21:15:01.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T21:15:01.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T21:15:01.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/bad-inc-map 1' TERM HUP INT 2026-02-17T21:15:01.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:20: run: TEST_bad_inc_map td/bad-inc-map 2026-02-17T21:15:01.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:26: TEST_bad_inc_map: local dir=td/bad-inc-map 2026-02-17T21:15:01.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:28: TEST_bad_inc_map: run_mon td/bad-inc-map a 2026-02-17T21:15:01.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/bad-inc-map 2026-02-17T21:15:01.351 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T21:15:01.351 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T21:15:01.351 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T21:15:01.351 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/bad-inc-map/a 2026-02-17T21:15:01.351 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/bad-inc-map/a --run-dir=td/bad-inc-map 2026-02-17T21:15:01.383 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T21:15:01.383 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:15:01.383 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:15:01.384 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:15:01.384 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:01.384 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.7918 2026-02-17T21:15:01.384 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.7918/$cluster-$name.asok' 2026-02-17T21:15:01.384 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/bad-inc-map/a '--log-file=td/bad-inc-map/$name.log' '--admin-socket=/tmp/ceph-asok.7918/$cluster-$name.asok' --mon-cluster-log-file=td/bad-inc-map/log --run-dir=td/bad-inc-map '--pid-file=td/bad-inc-map/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T21:15:01.413 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T21:15:01.416 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T21:15:01.416 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:15:01.416 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:15:01.416 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T21:15:01.416 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T21:15:01.417 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:15:01.417 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:15:01.417 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:15:01.417 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:15:01.417 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:01.417 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.7918 2026-02-17T21:15:01.417 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.7918/ceph-mon.a.asok 2026-02-17T21:15:01.417 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:15:01.418 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.7918/ceph-mon.a.asok config get fsid 2026-02-17T21:15:01.472 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T21:15:01.472 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:15:01.472 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:15:01.472 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T21:15:01.472 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T21:15:01.473 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:15:01.473 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:15:01.473 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:15:01.474 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:15:01.474 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:01.474 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.7918 2026-02-17T21:15:01.474 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.7918/ceph-mon.a.asok 2026-02-17T21:15:01.474 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:15:01.474 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.7918/ceph-mon.a.asok config get mon_host 2026-02-17T21:15:01.528 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:29: TEST_bad_inc_map: run_mgr td/bad-inc-map x 2026-02-17T21:15:01.528 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/bad-inc-map 2026-02-17T21:15:01.528 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T21:15:01.528 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T21:15:01.529 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T21:15:01.529 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/bad-inc-map/x 2026-02-17T21:15:01.529 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T21:15:01.639 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T21:15:01.639 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:15:01.639 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:15:01.640 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:15:01.640 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:01.640 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.7918 2026-02-17T21:15:01.640 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.7918/$cluster-$name.asok' 2026-02-17T21:15:01.640 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:15:01.641 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/bad-inc-map/x '--log-file=td/bad-inc-map/$name.log' '--admin-socket=/tmp/ceph-asok.7918/$cluster-$name.asok' --run-dir=td/bad-inc-map '--pid-file=td/bad-inc-map/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:15:01.663 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:30: TEST_bad_inc_map: run_osd td/bad-inc-map 0 2026-02-17T21:15:01.663 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/bad-inc-map 2026-02-17T21:15:01.663 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:15:01.663 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T21:15:01.664 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:15:01.664 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/bad-inc-map/0 2026-02-17T21:15:01.664 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=af30db47-6aab-48c3-a6a9-80b65597d8b3 --auth-supported=none --mon-host=127.0.0.1:52089 ' 2026-02-17T21:15:01.664 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:15:01.664 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:15:01.664 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:15:01.664 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/0' 2026-02-17T21:15:01.664 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/0/journal' 2026-02-17T21:15:01.664 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:15:01.664 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:15:01.664 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2026-02-17T21:15:01.664 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:15:01.665 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:15:01.665 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:15:01.665 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:15:01.665 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:01.665 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.7918 2026-02-17T21:15:01.665 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.7918/$cluster-$name.asok' 2026-02-17T21:15:01.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.7918/$cluster-$name.asok' 2026-02-17T21:15:01.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:15:01.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:15:01.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:15:01.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2026-02-17T21:15:01.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2026-02-17T21:15:01.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:15:01.666 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:15:01.666 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:15:01.666 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:15:01.666 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:15:01.666 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:15:01.666 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/bad-inc-map/0 2026-02-17T21:15:01.666 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:15:01.666 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b9b034a0-10f2-4a13-9794-c6a60f72d779 2026-02-17T21:15:01.666 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 b9b034a0-10f2-4a13-9794-c6a60f72d779' 2026-02-17T21:15:01.667 INFO:tasks.workunit.client.0.trial031.stdout:add osd0 b9b034a0-10f2-4a13-9794-c6a60f72d779 2026-02-17T21:15:01.667 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:15:01.677 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBV2pRpqMQGKBAA+6dcDdYMw4zuuBdcU6aYRQ== 2026-02-17T21:15:01.677 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBV2pRpqMQGKBAA+6dcDdYMw4zuuBdcU6aYRQ=="}' 2026-02-17T21:15:01.677 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b9b034a0-10f2-4a13-9794-c6a60f72d779 -i td/bad-inc-map/0/new.json 2026-02-17T21:15:01.784 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:15:01.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/bad-inc-map/0/new.json 2026-02-17T21:15:01.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=af30db47-6aab-48c3-a6a9-80b65597d8b3 --auth-supported=none --mon-host=127.0.0.1:52089 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/0 --osd-journal=td/bad-inc-map/0/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.7918/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBV2pRpqMQGKBAA+6dcDdYMw4zuuBdcU6aYRQ== --osd-uuid b9b034a0-10f2-4a13-9794-c6a60f72d779 2026-02-17T21:15:01.808 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:01.799+0000 7f3e378aa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:01.809 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:01.803+0000 7f3e378aa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:01.810 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:01.803+0000 7f3e378aa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:01.810 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:01.803+0000 7f3e378aa8c0 -1 bdev(0x558bf1f0fc00 td/bad-inc-map/0/block) open stat got: (1) Operation not permitted 2026-02-17T21:15:01.810 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:01.803+0000 7f3e378aa8c0 -1 bluestore(td/bad-inc-map/0) _read_fsid unparsable uuid 2026-02-17T21:15:04.569 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/bad-inc-map/0/keyring 2026-02-17T21:15:04.569 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:15:04.570 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T21:15:04.570 INFO:tasks.workunit.client.0.trial031.stdout:adding osd0 key to auth repository 2026-02-17T21:15:04.570 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/bad-inc-map/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:15:04.847 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:15:04.848 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T21:15:04.848 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=af30db47-6aab-48c3-a6a9-80b65597d8b3 --auth-supported=none --mon-host=127.0.0.1:52089 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/0 --osd-journal=td/bad-inc-map/0/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.7918/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:15:04.848 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:15:04.848 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:15:04.848 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:15:04.862 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:04.855+0000 7f3bf1c738c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:04.863 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:04.855+0000 7f3bf1c738c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:04.864 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:04.855+0000 7f3bf1c738c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:05.052 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T21:15:05.053 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:15:05.053 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:15:05.053 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:15:05.053 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:15:05.053 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:05.053 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:15:05.054 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:15:05.054 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:05.054 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:15:05.247 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:06.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:06.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:06.249 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:15:06.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:15:06.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:06.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:15:06.358 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:06.351+0000 7f3bf1c738c0 -1 Falling back to public interface 2026-02-17T21:15:06.444 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:07.446 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:07.446 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:07.446 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:15:07.446 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:15:07.447 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:07.447 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:15:07.638 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:08.639 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:08.639 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:08.640 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:15:08.640 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:15:08.641 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:08.641 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:15:08.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:09.541 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:09.535+0000 7f3bf1c738c0 -1 osd.0 0 log_to_monitors true 2026-02-17T21:15:09.832 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:09.832 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:09.832 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:15:09.833 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:15:09.833 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:09.833 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:15:10.036 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:11.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:11.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:11.038 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:15:11.038 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:15:11.039 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:11.039 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:15:11.233 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/184058240,v1:127.0.0.1:6803/184058240] [v2:127.0.0.1:6804/184058240,v1:127.0.0.1:6805/184058240] exists,up b9b034a0-10f2-4a13-9794-c6a60f72d779 2026-02-17T21:15:11.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:15:11.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:15:11.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:15:11.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:31: TEST_bad_inc_map: run_osd td/bad-inc-map 1 2026-02-17T21:15:11.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/bad-inc-map 2026-02-17T21:15:11.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:15:11.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T21:15:11.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:15:11.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/bad-inc-map/1 2026-02-17T21:15:11.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=af30db47-6aab-48c3-a6a9-80b65597d8b3 --auth-supported=none --mon-host=127.0.0.1:52089 ' 2026-02-17T21:15:11.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:15:11.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:15:11.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:15:11.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/1' 2026-02-17T21:15:11.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/1/journal' 2026-02-17T21:15:11.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:15:11.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:15:11.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2026-02-17T21:15:11.238 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:15:11.238 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:15:11.239 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:15:11.239 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:15:11.239 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:11.239 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.7918 2026-02-17T21:15:11.239 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.7918/$cluster-$name.asok' 2026-02-17T21:15:11.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.7918/$cluster-$name.asok' 2026-02-17T21:15:11.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:15:11.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:15:11.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:15:11.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2026-02-17T21:15:11.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2026-02-17T21:15:11.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:15:11.240 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:15:11.240 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:15:11.240 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:15:11.240 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:15:11.240 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:15:11.240 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/bad-inc-map/1 2026-02-17T21:15:11.240 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:15:11.241 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=67d1419e-f0b0-40aa-9397-412501d537e1 2026-02-17T21:15:11.241 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 67d1419e-f0b0-40aa-9397-412501d537e1' 2026-02-17T21:15:11.241 INFO:tasks.workunit.client.0.trial031.stdout:add osd1 67d1419e-f0b0-40aa-9397-412501d537e1 2026-02-17T21:15:11.241 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:15:11.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBf2pRphgiODhAAUsJdjd/4M7rnc6pT68Vtig== 2026-02-17T21:15:11.250 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBf2pRphgiODhAAUsJdjd/4M7rnc6pT68Vtig=="}' 2026-02-17T21:15:11.250 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 67d1419e-f0b0-40aa-9397-412501d537e1 -i td/bad-inc-map/1/new.json 2026-02-17T21:15:11.451 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:15:11.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/bad-inc-map/1/new.json 2026-02-17T21:15:11.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=af30db47-6aab-48c3-a6a9-80b65597d8b3 --auth-supported=none --mon-host=127.0.0.1:52089 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/1 --osd-journal=td/bad-inc-map/1/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.7918/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBf2pRphgiODhAAUsJdjd/4M7rnc6pT68Vtig== --osd-uuid 67d1419e-f0b0-40aa-9397-412501d537e1 2026-02-17T21:15:11.476 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:11.467+0000 7fe8ee5238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:11.477 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:11.471+0000 7fe8ee5238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:11.478 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:11.471+0000 7fe8ee5238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:11.478 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:11.471+0000 7fe8ee5238c0 -1 bdev(0x5558dd307c00 td/bad-inc-map/1/block) open stat got: (1) Operation not permitted 2026-02-17T21:15:11.479 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:11.471+0000 7fe8ee5238c0 -1 bluestore(td/bad-inc-map/1) _read_fsid unparsable uuid 2026-02-17T21:15:13.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/bad-inc-map/1/keyring 2026-02-17T21:15:13.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:15:13.793 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T21:15:13.793 INFO:tasks.workunit.client.0.trial031.stdout:adding osd1 key to auth repository 2026-02-17T21:15:13.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/bad-inc-map/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:15:14.057 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T21:15:14.058 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:15:14.058 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=af30db47-6aab-48c3-a6a9-80b65597d8b3 --auth-supported=none --mon-host=127.0.0.1:52089 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/1 --osd-journal=td/bad-inc-map/1/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.7918/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:15:14.059 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:15:14.059 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:15:14.059 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:15:14.078 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:14.071+0000 7f27380058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:14.080 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:14.071+0000 7f27380058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:14.081 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:14.075+0000 7f27380058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:14.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T21:15:14.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:15:14.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:15:14.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:15:14.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:15:14.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:14.259 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:15:14.260 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:15:14.260 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:14.260 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:15:14.456 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:15.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:15.459 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:15.459 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:15:15.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:15:15.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:15.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:15:15.590 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:15.583+0000 7f27380058c0 -1 Falling back to public interface 2026-02-17T21:15:15.656 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:16.657 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:16.657 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:16.658 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:15:16.658 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:15:16.658 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:16.659 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:15:16.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:17.851 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:17.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:17.852 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:15:17.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:15:17.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:17.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:15:18.056 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:18.581 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:18.575+0000 7f27380058c0 -1 osd.1 0 log_to_monitors true 2026-02-17T21:15:19.058 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:19.058 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:19.059 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:15:19.059 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:15:19.059 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:19.059 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:15:19.257 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3036080107,v1:127.0.0.1:6811/3036080107] [v2:127.0.0.1:6812/3036080107,v1:127.0.0.1:6813/3036080107] exists,up 67d1419e-f0b0-40aa-9397-412501d537e1 2026-02-17T21:15:19.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:15:19.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:15:19.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:15:19.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:32: TEST_bad_inc_map: run_osd td/bad-inc-map 2 2026-02-17T21:15:19.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/bad-inc-map 2026-02-17T21:15:19.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:15:19.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T21:15:19.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:15:19.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/bad-inc-map/2 2026-02-17T21:15:19.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=af30db47-6aab-48c3-a6a9-80b65597d8b3 --auth-supported=none --mon-host=127.0.0.1:52089 ' 2026-02-17T21:15:19.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:15:19.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:15:19.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:15:19.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/2' 2026-02-17T21:15:19.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/2/journal' 2026-02-17T21:15:19.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:15:19.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:15:19.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2026-02-17T21:15:19.259 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:15:19.259 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:15:19.259 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:15:19.260 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:15:19.260 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:19.260 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.7918 2026-02-17T21:15:19.260 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.7918/$cluster-$name.asok' 2026-02-17T21:15:19.260 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.7918/$cluster-$name.asok' 2026-02-17T21:15:19.260 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:15:19.260 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:15:19.260 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:15:19.260 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2026-02-17T21:15:19.260 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2026-02-17T21:15:19.260 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:15:19.260 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:15:19.260 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:15:19.260 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:15:19.260 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:15:19.261 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:15:19.261 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/bad-inc-map/2 2026-02-17T21:15:19.261 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:15:19.261 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=65973fa1-48b2-4b6f-a502-bdbd578c1add 2026-02-17T21:15:19.261 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 65973fa1-48b2-4b6f-a502-bdbd578c1add' 2026-02-17T21:15:19.261 INFO:tasks.workunit.client.0.trial031.stdout:add osd2 65973fa1-48b2-4b6f-a502-bdbd578c1add 2026-02-17T21:15:19.262 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:15:19.273 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBn2pRpExv7DxAAFldzeyToaSRWg1+Ddf2vbA== 2026-02-17T21:15:19.273 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBn2pRpExv7DxAAFldzeyToaSRWg1+Ddf2vbA=="}' 2026-02-17T21:15:19.273 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 65973fa1-48b2-4b6f-a502-bdbd578c1add -i td/bad-inc-map/2/new.json 2026-02-17T21:15:19.478 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:15:19.487 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/bad-inc-map/2/new.json 2026-02-17T21:15:19.488 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=af30db47-6aab-48c3-a6a9-80b65597d8b3 --auth-supported=none --mon-host=127.0.0.1:52089 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/2 --osd-journal=td/bad-inc-map/2/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.7918/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBn2pRpExv7DxAAFldzeyToaSRWg1+Ddf2vbA== --osd-uuid 65973fa1-48b2-4b6f-a502-bdbd578c1add 2026-02-17T21:15:19.504 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:19.494+0000 7f3b352dc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:19.505 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:19.498+0000 7f3b352dc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:19.506 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:19.498+0000 7f3b352dc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:19.506 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:19.498+0000 7f3b352dc8c0 -1 bdev(0x56310fd27c00 td/bad-inc-map/2/block) open stat got: (1) Operation not permitted 2026-02-17T21:15:19.506 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:19.498+0000 7f3b352dc8c0 -1 bluestore(td/bad-inc-map/2) _read_fsid unparsable uuid 2026-02-17T21:15:21.809 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/bad-inc-map/2/keyring 2026-02-17T21:15:21.809 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:15:21.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T21:15:21.810 INFO:tasks.workunit.client.0.trial031.stdout:adding osd2 key to auth repository 2026-02-17T21:15:21.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/bad-inc-map/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:15:22.082 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T21:15:22.082 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:15:22.083 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=af30db47-6aab-48c3-a6a9-80b65597d8b3 --auth-supported=none --mon-host=127.0.0.1:52089 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/2 --osd-journal=td/bad-inc-map/2/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.7918/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:15:22.083 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:15:22.083 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:15:22.084 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:15:22.103 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:22.094+0000 7f7e9fccc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:22.105 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:22.098+0000 7f7e9fccc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:22.106 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:22.098+0000 7f7e9fccc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:22.282 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T21:15:22.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:15:22.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:15:22.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:15:22.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:15:22.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:22.283 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:15:22.284 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:15:22.284 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:22.284 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:15:22.480 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:23.482 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:23.482 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:23.482 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:15:23.483 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:15:23.483 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:23.483 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:15:23.658 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:23.650+0000 7f7e9fccc8c0 -1 Falling back to public interface 2026-02-17T21:15:23.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:24.677 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:24.677 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:24.678 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:15:24.678 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:15:24.679 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:24.679 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:15:24.872 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:25.873 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:25.873 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:25.873 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:15:25.873 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:15:25.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:25.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:15:26.066 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:26.647 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:26.638+0000 7f7e9fccc8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T21:15:27.067 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:27.068 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:27.068 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:15:27.068 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:15:27.068 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:27.068 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:15:27.262 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:28.263 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:28.264 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:28.264 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:15:28.264 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:15:28.265 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:28.265 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:15:28.461 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4125689181,v1:127.0.0.1:6819/4125689181] [v2:127.0.0.1:6820/4125689181,v1:127.0.0.1:6821/4125689181] exists,up 65973fa1-48b2-4b6f-a502-bdbd578c1add 2026-02-17T21:15:28.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:15:28.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:15:28.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:15:28.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:34: TEST_bad_inc_map: ceph config set osd.2 osd_inject_bad_map_crc_probability 1 2026-02-17T21:15:28.657 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:37: TEST_bad_inc_map: create_pool foo 8 2026-02-17T21:15:28.657 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 8 2026-02-17T21:15:28.915 INFO:tasks.workunit.client.0.trial031.stderr:pool 'foo' already exists 2026-02-17T21:15:28.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T21:15:29.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:38: TEST_bad_inc_map: ceph osd pool set foo min_size 1 2026-02-17T21:15:30.218 INFO:tasks.workunit.client.0.trial031.stderr:set pool 1 min_size to 1 2026-02-17T21:15:30.228 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:39: TEST_bad_inc_map: ceph osd pool set foo min_size 2 2026-02-17T21:15:30.526 INFO:tasks.workunit.client.0.trial031.stderr:set pool 1 min_size to 2 2026-02-17T21:15:30.536 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:41: TEST_bad_inc_map: sleep 5 2026-02-17T21:15:35.538 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:44: TEST_bad_inc_map: TIMEOUT=10 2026-02-17T21:15:35.538 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:44: TEST_bad_inc_map: wait_for_osd up 0 2026-02-17T21:15:35.539 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:15:35.539 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:15:35.539 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:15:35.539 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:15:35.539 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 10 )) 2026-02-17T21:15:35.539 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:15:35.540 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:15:35.540 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:35.540 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:15:35.733 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 5 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/184058240,v1:127.0.0.1:6803/184058240] [v2:127.0.0.1:6804/184058240,v1:127.0.0.1:6805/184058240] exists,up b9b034a0-10f2-4a13-9794-c6a60f72d779 2026-02-17T21:15:35.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:15:35.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:15:35.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:15:35.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:45: TEST_bad_inc_map: TIMEOUT=10 2026-02-17T21:15:35.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:45: TEST_bad_inc_map: wait_for_osd up 1 2026-02-17T21:15:35.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:15:35.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:15:35.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:15:35.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:15:35.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 10 )) 2026-02-17T21:15:35.735 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:15:35.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:15:35.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:35.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:15:35.928 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 9 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3036080107,v1:127.0.0.1:6811/3036080107] [v2:127.0.0.1:6812/3036080107,v1:127.0.0.1:6813/3036080107] exists,up 67d1419e-f0b0-40aa-9397-412501d537e1 2026-02-17T21:15:35.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:15:35.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:15:35.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:15:35.930 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:46: TEST_bad_inc_map: TIMEOUT=10 2026-02-17T21:15:35.930 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:46: TEST_bad_inc_map: wait_for_osd up 2 2026-02-17T21:15:35.930 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:15:35.930 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:15:35.930 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:15:35.930 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:15:35.930 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 10 )) 2026-02-17T21:15:35.930 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:15:35.930 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:15:35.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:35.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:15:36.124 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4125689181,v1:127.0.0.1:6819/4125689181] [v2:127.0.0.1:6820/4125689181,v1:127.0.0.1:6821/4125689181] exists,up 65973fa1-48b2-4b6f-a502-bdbd578c1add 2026-02-17T21:15:36.125 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:15:36.125 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:15:36.125 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:15:36.126 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:49: TEST_bad_inc_map: grep 'injecting map crc failure' td/bad-inc-map/osd.2.log 2026-02-17T21:15:36.127 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:28.862+0000 7f7e8d321640 -1 osd.2 13 handle_osd_map injecting map crc failure 2026-02-17T21:15:36.127 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:28.962+0000 7f7e8f325640 -1 osd.2 14 handle_osd_map injecting map crc failure 2026-02-17T21:15:36.127 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:28.966+0000 7f7e8f325640 -1 osd.2 14 handle_osd_map injecting map crc failure 2026-02-17T21:15:36.127 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:30.106+0000 7f7e8f325640 -1 osd.2 15 handle_osd_map injecting map crc failure 2026-02-17T21:15:36.127 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:30.210+0000 7f7e8f325640 -1 osd.2 17 handle_osd_map injecting map crc failure 2026-02-17T21:15:36.127 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:30.210+0000 7f7e8d321640 -1 osd.2 17 handle_osd_map injecting map crc failure 2026-02-17T21:15:36.127 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:30.210+0000 7f7e8d321640 -1 osd.2 17 handle_osd_map injecting map crc failure 2026-02-17T21:15:36.127 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:30.414+0000 7f7e8f325640 -1 osd.2 18 handle_osd_map injecting map crc failure 2026-02-17T21:15:36.127 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:30.518+0000 7f7e8f325640 -1 osd.2 19 handle_osd_map injecting map crc failure 2026-02-17T21:15:36.127 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:30.518+0000 7f7e8d321640 -1 osd.2 19 handle_osd_map injecting map crc failure 2026-02-17T21:15:36.128 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:50: TEST_bad_inc_map: grep 'bailing because last' td/bad-inc-map/osd.2.log 2026-02-17T21:15:36.129 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:28.862+0000 7f7e8d321640 10 osd.2 13 handle_osd_map bailing because last < start (13<14) 2026-02-17T21:15:36.129 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:28.962+0000 7f7e8f325640 10 osd.2 14 handle_osd_map bailing because last < start (14<15) 2026-02-17T21:15:36.129 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:28.966+0000 7f7e8f325640 10 osd.2 14 handle_osd_map bailing because last < start (14<15) 2026-02-17T21:15:36.129 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:30.106+0000 7f7e8f325640 10 osd.2 15 handle_osd_map bailing because last < start (15<16) 2026-02-17T21:15:36.129 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:30.210+0000 7f7e8f325640 10 osd.2 17 handle_osd_map bailing because last < start (17<18) 2026-02-17T21:15:36.129 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:30.210+0000 7f7e8d321640 10 osd.2 17 handle_osd_map bailing because last < start (17<18) 2026-02-17T21:15:36.129 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:30.214+0000 7f7e8d321640 10 osd.2 17 handle_osd_map bailing because last < start (17<18) 2026-02-17T21:15:36.129 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:30.414+0000 7f7e8f325640 10 osd.2 18 handle_osd_map bailing because last < start (18<19) 2026-02-17T21:15:36.130 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:30.518+0000 7f7e8f325640 10 osd.2 19 handle_osd_map bailing because last < start (19<20) 2026-02-17T21:15:36.130 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:15:30.518+0000 7f7e8d321640 10 osd.2 19 handle_osd_map bailing because last < start (19<20) 2026-02-17T21:15:36.130 INFO:tasks.workunit.client.0.trial031.stdout:success 2026-02-17T21:15:36.130 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:52: TEST_bad_inc_map: echo success 2026-02-17T21:15:36.130 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:54: TEST_bad_inc_map: delete_pool foo 2026-02-17T21:15:36.131 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=foo 2026-02-17T21:15:36.131 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete foo foo --yes-i-really-really-mean-it 2026-02-17T21:15:36.368 INFO:tasks.workunit.client.0.trial031.stderr:pool 'foo' does not exist 2026-02-17T21:15:36.377 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:55: TEST_bad_inc_map: kill_daemons td/bad-inc-map 2026-02-17T21:15:36.378 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:15:36.378 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:15:36.378 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:15:36.378 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:15:36.378 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:15:44.694 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:15:44.694 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:21: run: teardown td/bad-inc-map 2026-02-17T21:15:44.694 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/bad-inc-map 2026-02-17T21:15:44.694 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:15:44.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/bad-inc-map KILL 2026-02-17T21:15:44.695 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:15:44.695 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:15:44.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:15:44.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:15:44.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:15:44.700 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:15:44.701 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:15:44.701 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:15:44.702 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:15:44.703 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:15:44.703 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:15:44.703 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:15:44.704 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:15:44.704 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:15:44.705 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:15:44.705 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:15:44.706 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:15:44.706 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:15:44.707 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/bad-inc-map 2026-02-17T21:15:44.717 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:15:44.717 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:44.717 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.7918 2026-02-17T21:15:44.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.7918 2026-02-17T21:15:44.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:15:44.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:15:44.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T21:15:44.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/bad-inc-map 0 2026-02-17T21:15:44.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/bad-inc-map 2026-02-17T21:15:44.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T21:15:44.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/bad-inc-map KILL 2026-02-17T21:15:44.720 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:15:44.720 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:15:44.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:15:44.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:15:44.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:15:44.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:15:44.722 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:15:44.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:15:44.723 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:15:44.724 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:15:44.724 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:15:44.724 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:15:44.725 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:15:44.726 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:15:44.726 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:15:44.726 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:15:44.727 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:15:44.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T21:15:44.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/bad-inc-map 2026-02-17T21:15:44.730 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:15:44.730 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:44.730 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.7918 2026-02-17T21:15:44.730 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.7918 2026-02-17T21:15:44.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:15:44.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:15:44.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T21:15:44.732 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T21:15:44.732 DEBUG:teuthology.orchestra.run.trial031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T21:15:44.785 INFO:tasks.workunit:Running workunit osd/divergent-priors.sh... 2026-02-17T21:15:44.785 DEBUG:teuthology.orchestra.run.trial031:workunit test osd/divergent-priors.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d855f53b89fdcec760fd9232a5fb55ed4fb111a1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh 2026-02-17T21:15:44.841 INFO:tasks.workunit.client.0.trial031.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T21:15:44.846 INFO:tasks.workunit.client.0.trial031.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T21:15:44.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T21:15:44.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T21:15:44.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T21:15:44.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T21:15:44.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T21:15:44.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T21:15:44.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T21:15:44.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T21:15:44.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/divergent-priors 2026-02-17T21:15:44.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:21: run: local dir=td/divergent-priors 2026-02-17T21:15:44.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:22: run: shift 2026-02-17T21:15:44.848 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:25: run: export loglen=12 2026-02-17T21:15:44.848 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:25: run: loglen=12 2026-02-17T21:15:44.848 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:26: run: export divisor=3 2026-02-17T21:15:44.848 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:26: run: divisor=3 2026-02-17T21:15:44.848 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: expr 12 / 2 2026-02-17T21:15:44.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: export trim=6 2026-02-17T21:15:44.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: trim=6 2026-02-17T21:15:44.849 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: expr 6 / 3 2026-02-17T21:15:44.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: export DIVERGENT_WRITE=2 2026-02-17T21:15:44.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: DIVERGENT_WRITE=2 2026-02-17T21:15:44.849 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: expr 6 / 3 2026-02-17T21:15:44.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: export DIVERGENT_REMOVE=2 2026-02-17T21:15:44.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: DIVERGENT_REMOVE=2 2026-02-17T21:15:44.851 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: expr 6 / 3 2026-02-17T21:15:44.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: export DIVERGENT_CREATE=2 2026-02-17T21:15:44.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: DIVERGENT_CREATE=2 2026-02-17T21:15:44.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:31: run: export poolname=test 2026-02-17T21:15:44.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:31: run: poolname=test 2026-02-17T21:15:44.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:32: run: export testobjects=100 2026-02-17T21:15:44.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:32: run: testobjects=100 2026-02-17T21:15:44.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:34: run: export CEPH_MON=127.0.0.1:7115 2026-02-17T21:15:44.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:34: run: CEPH_MON=127.0.0.1:7115 2026-02-17T21:15:44.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:35: run: export CEPH_ARGS 2026-02-17T21:15:44.853 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:36: run: uuidgen 2026-02-17T21:15:44.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:36: run: CEPH_ARGS+='--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none ' 2026-02-17T21:15:44.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:37: run: CEPH_ARGS+='--mon-host=127.0.0.1:7115 ' 2026-02-17T21:15:44.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:39: run: CEPH_ARGS+='--osd_force_auth_primary_missing_objects=1000000 ' 2026-02-17T21:15:44.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:40: run: CEPH_ARGS+='--osd_debug_pg_log_writeout=true ' 2026-02-17T21:15:44.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:41: run: CEPH_ARGS+='--osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:15:44.854 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: set 2026-02-17T21:15:44.854 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T21:15:44.857 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: local 'funcs=TEST_divergent 2026-02-17T21:15:44.857 INFO:tasks.workunit.client.0.trial031.stderr:TEST_divergent_2 2026-02-17T21:15:44.857 INFO:tasks.workunit.client.0.trial031.stderr:TEST_divergent_3 2026-02-17T21:15:44.857 INFO:tasks.workunit.client.0.trial031.stderr:TEST_divergent_ec' 2026-02-17T21:15:44.857 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-17T21:15:44.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-17T21:15:44.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2026-02-17T21:15:44.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2026-02-17T21:15:44.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-17T21:15:44.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:15:44.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-17T21:15:44.858 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:15:44.858 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:15:44.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:15:44.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:15:44.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:15:44.859 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:15:44.859 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:15:44.860 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:15:44.861 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:15:44.862 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:15:44.862 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:15:44.863 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:15:44.863 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:15:44.864 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:15:44.864 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:15:44.864 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:15:44.865 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:15:44.866 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:15:44.866 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-17T21:15:44.868 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:15:44.868 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:44.868 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:15:44.868 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.16076 2026-02-17T21:15:44.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:15:44.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:15:44.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2026-02-17T21:15:44.871 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T21:15:44.871 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:44.871 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:15:44.871 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.16076 2026-02-17T21:15:44.873 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T21:15:44.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T21:15:44.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T21:15:44.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T21:15:44.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-17T21:15:44.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent td/divergent-priors 2026-02-17T21:15:44.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:57: TEST_divergent: local dir=td/divergent-priors 2026-02-17T21:15:44.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:60: TEST_divergent: local dummyfile=/etc/fstab 2026-02-17T21:15:44.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:61: TEST_divergent: local dummyfile2=/etc/resolv.conf 2026-02-17T21:15:44.875 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:63: TEST_divergent: local num_osds=3 2026-02-17T21:15:44.875 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: expr 3 - 1 2026-02-17T21:15:44.875 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: seq 0 2 2026-02-17T21:15:44.876 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: local 'osds=0 2026-02-17T21:15:44.876 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:15:44.877 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:15:44.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:65: TEST_divergent: run_mon td/divergent-priors a 2026-02-17T21:15:44.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2026-02-17T21:15:44.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T21:15:44.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T21:15:44.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T21:15:44.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2026-02-17T21:15:44.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-17T21:15:44.900 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T21:15:44.900 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:15:44.900 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:15:44.900 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:15:44.900 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:44.900 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:15:44.901 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:15:44.901 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T21:15:44.930 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T21:15:44.932 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T21:15:44.932 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:15:44.932 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:15:44.932 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T21:15:44.932 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T21:15:44.933 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:15:44.933 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:15:44.933 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:15:44.933 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:15:44.934 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:44.934 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:15:44.934 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-mon.a.asok 2026-02-17T21:15:44.934 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:15:44.934 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.16076/ceph-mon.a.asok config get fsid 2026-02-17T21:15:44.988 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T21:15:44.988 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:15:44.989 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:15:44.989 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T21:15:44.989 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T21:15:44.989 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:15:44.989 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:15:44.989 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:15:44.990 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:15:44.990 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:44.990 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:15:44.990 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-mon.a.asok 2026-02-17T21:15:44.990 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:15:44.990 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.16076/ceph-mon.a.asok config get mon_host 2026-02-17T21:15:45.045 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:66: TEST_divergent: run_mgr td/divergent-priors x 2026-02-17T21:15:45.045 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/divergent-priors 2026-02-17T21:15:45.045 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T21:15:45.045 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T21:15:45.045 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T21:15:45.045 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/divergent-priors/x 2026-02-17T21:15:45.045 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T21:15:45.161 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T21:15:45.161 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:15:45.161 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:15:45.161 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:15:45.161 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:45.161 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:15:45.161 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:15:45.162 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:15:45.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:15:45.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:67: TEST_divergent: for i in $osds 2026-02-17T21:15:45.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:69: TEST_divergent: run_osd td/divergent-priors 0 2026-02-17T21:15:45.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-17T21:15:45.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:15:45.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T21:15:45.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:15:45.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/0 2026-02-17T21:15:45.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:15:45.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:15:45.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:15:45.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:15:45.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-17T21:15:45.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-17T21:15:45.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:15:45.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:15:45.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:15:45.187 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:15:45.187 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:15:45.187 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:15:45.187 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:15:45.187 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:45.187 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:15:45.187 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:15:45.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:15:45.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:15:45.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:15:45.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:15:45.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:15:45.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:15:45.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:15:45.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:15:45.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:15:45.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:15:45.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:15:45.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:15:45.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/0 2026-02-17T21:15:45.188 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:15:45.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=f21d462f-1bd6-4e5f-afc1-346559015142 2026-02-17T21:15:45.189 INFO:tasks.workunit.client.0.trial031.stdout:add osd0 f21d462f-1bd6-4e5f-afc1-346559015142 2026-02-17T21:15:45.190 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 f21d462f-1bd6-4e5f-afc1-346559015142' 2026-02-17T21:15:45.190 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:15:45.199 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCB2pRpYwWZCxAAW2kn4T8k9d550YQtJdeMhg== 2026-02-17T21:15:45.200 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCB2pRpYwWZCxAAW2kn4T8k9d550YQtJdeMhg=="}' 2026-02-17T21:15:45.200 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new f21d462f-1bd6-4e5f-afc1-346559015142 -i td/divergent-priors/0/new.json 2026-02-17T21:15:45.311 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:15:45.319 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/0/new.json 2026-02-17T21:15:45.320 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCB2pRpYwWZCxAAW2kn4T8k9d550YQtJdeMhg== --osd-uuid f21d462f-1bd6-4e5f-afc1-346559015142 2026-02-17T21:15:45.339 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:45.330+0000 7efdbaf938c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:45.341 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:45.334+0000 7efdbaf938c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:45.342 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:45.334+0000 7efdbaf938c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:45.342 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:45.334+0000 7efdbaf938c0 -1 bdev(0x55d689e51c00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-17T21:15:45.342 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:45.334+0000 7efdbaf938c0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-17T21:15:48.125 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-17T21:15:48.125 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:15:48.126 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T21:15:48.126 INFO:tasks.workunit.client.0.trial031.stdout:adding osd0 key to auth repository 2026-02-17T21:15:48.126 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:15:48.409 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:15:48.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T21:15:48.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:15:48.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:15:48.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:15:48.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:15:48.428 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:48.418+0000 7fe9efb448c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:48.429 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:48.422+0000 7fe9efb448c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:48.430 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:48.422+0000 7fe9efb448c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:48.617 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T21:15:48.618 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:15:48.618 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:15:48.618 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:15:48.618 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:15:48.618 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:48.618 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:15:48.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:15:48.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:48.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:15:48.819 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:49.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:49.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:49.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:15:49.821 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:15:49.822 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:49.822 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:15:50.028 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:50.166 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:50.158+0000 7fe9efb448c0 -1 Falling back to public interface 2026-02-17T21:15:51.030 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:51.030 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:51.030 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:15:51.031 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:15:51.031 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:51.031 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:15:51.236 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:52.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:52.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:52.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:15:52.238 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:15:52.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:52.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:15:52.440 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:53.148 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:53.142+0000 7fe9efb448c0 -1 osd.0 0 log_to_monitors true 2026-02-17T21:15:53.441 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:53.442 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:53.442 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:15:53.443 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:15:53.443 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:53.443 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:15:53.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:54.645 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:54.646 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:54.646 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:15:54.647 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:15:54.647 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:54.647 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:15:54.841 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3884033888,v1:127.0.0.1:6803/3884033888] [v2:127.0.0.1:6804/3884033888,v1:127.0.0.1:6805/3884033888] exists,up f21d462f-1bd6-4e5f-afc1-346559015142 2026-02-17T21:15:54.844 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:15:54.844 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:15:54.844 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:15:54.844 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:67: TEST_divergent: for i in $osds 2026-02-17T21:15:54.844 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:69: TEST_divergent: run_osd td/divergent-priors 1 2026-02-17T21:15:54.845 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-17T21:15:54.845 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:15:54.845 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T21:15:54.845 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:15:54.845 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/1 2026-02-17T21:15:54.845 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:15:54.845 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:15:54.845 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:15:54.845 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:15:54.845 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-17T21:15:54.845 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-17T21:15:54.845 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:15:54.846 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:15:54.846 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:15:54.846 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:15:54.846 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:15:54.846 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:15:54.846 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:15:54.846 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:15:54.846 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:15:54.846 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:15:54.846 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:15:54.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:15:54.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:15:54.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:15:54.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:15:54.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:15:54.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:15:54.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:15:54.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:15:54.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:15:54.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:15:54.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:15:54.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/1 2026-02-17T21:15:54.847 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:15:54.848 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ee415559-573a-4cec-9568-aa72c415cf6a 2026-02-17T21:15:54.848 INFO:tasks.workunit.client.0.trial031.stdout:add osd1 ee415559-573a-4cec-9568-aa72c415cf6a 2026-02-17T21:15:54.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 ee415559-573a-4cec-9568-aa72c415cf6a' 2026-02-17T21:15:54.849 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:15:54.859 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCK2pRp5qbtMhAAZrdHWzpMYH0fhuITdj0iBQ== 2026-02-17T21:15:54.860 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCK2pRp5qbtMhAAZrdHWzpMYH0fhuITdj0iBQ=="}' 2026-02-17T21:15:54.860 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ee415559-573a-4cec-9568-aa72c415cf6a -i td/divergent-priors/1/new.json 2026-02-17T21:15:55.070 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:15:55.079 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/1/new.json 2026-02-17T21:15:55.080 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCK2pRp5qbtMhAAZrdHWzpMYH0fhuITdj0iBQ== --osd-uuid ee415559-573a-4cec-9568-aa72c415cf6a 2026-02-17T21:15:55.099 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:55.090+0000 7f56f8a658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:55.100 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:55.094+0000 7f56f8a658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:55.102 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:55.094+0000 7f56f8a658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:55.102 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:55.094+0000 7f56f8a658c0 -1 bdev(0x56293536bc00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-17T21:15:55.102 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:55.094+0000 7f56f8a658c0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-17T21:15:57.365 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-17T21:15:57.366 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:15:57.366 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T21:15:57.367 INFO:tasks.workunit.client.0.trial031.stdout:adding osd1 key to auth repository 2026-02-17T21:15:57.367 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:15:57.635 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:15:57.636 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T21:15:57.637 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:15:57.637 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:15:57.637 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:15:57.637 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:15:57.665 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:57.658+0000 7fa994d278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:57.666 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:57.658+0000 7fa994d278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:57.667 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:57.658+0000 7fa994d278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:15:57.842 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T21:15:57.842 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:15:57.842 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:15:57.842 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:15:57.842 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:15:57.843 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:57.843 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:15:57.843 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:15:57.844 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:57.844 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:15:58.045 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:15:59.046 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:15:59.046 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:15:59.047 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:15:59.047 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:15:59.048 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:15:59.048 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:15:59.162 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:15:59.153+0000 7fa994d278c0 -1 Falling back to public interface 2026-02-17T21:15:59.246 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:00.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:00.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:00.248 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:16:00.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:16:00.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:00.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:16:00.449 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:01.450 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:01.450 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:01.451 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:16:01.451 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:16:01.452 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:01.452 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:16:01.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:02.131 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:02.121+0000 7fa994d278c0 -1 osd.1 0 log_to_monitors true 2026-02-17T21:16:02.645 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:02.645 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:02.646 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:16:02.646 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:16:02.647 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:02.647 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:16:02.849 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2275153387,v1:127.0.0.1:6811/2275153387] [v2:127.0.0.1:6812/2275153387,v1:127.0.0.1:6813/2275153387] exists,up ee415559-573a-4cec-9568-aa72c415cf6a 2026-02-17T21:16:02.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:16:02.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:16:02.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:16:02.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:67: TEST_divergent: for i in $osds 2026-02-17T21:16:02.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:69: TEST_divergent: run_osd td/divergent-priors 2 2026-02-17T21:16:02.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-17T21:16:02.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:16:02.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T21:16:02.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:16:02.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/2 2026-02-17T21:16:02.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:16:02.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:16:02.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:16:02.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:16:02.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-17T21:16:02.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-17T21:16:02.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:16:02.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:16:02.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:16:02.855 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:16:02.855 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:16:02.855 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:16:02.855 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:16:02.855 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:16:02.855 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:16:02.855 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:16:02.855 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:16:02.855 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:16:02.855 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:16:02.855 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:16:02.855 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:16:02.855 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:16:02.855 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:16:02.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:16:02.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:16:02.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:16:02.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:16:02.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:16:02.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/2 2026-02-17T21:16:02.857 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:16:02.857 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=f52fba25-8021-4172-b9c1-88d5728ad329 2026-02-17T21:16:02.857 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 f52fba25-8021-4172-b9c1-88d5728ad329' 2026-02-17T21:16:02.857 INFO:tasks.workunit.client.0.trial031.stdout:add osd2 f52fba25-8021-4172-b9c1-88d5728ad329 2026-02-17T21:16:02.858 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:16:02.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCS2pRpoCJ6MxAACm98JX+9/IU2XwCSysKecA== 2026-02-17T21:16:02.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCS2pRpoCJ6MxAACm98JX+9/IU2XwCSysKecA=="}' 2026-02-17T21:16:02.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new f52fba25-8021-4172-b9c1-88d5728ad329 -i td/divergent-priors/2/new.json 2026-02-17T21:16:03.074 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:16:03.083 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/2/new.json 2026-02-17T21:16:03.084 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCS2pRpoCJ6MxAACm98JX+9/IU2XwCSysKecA== --osd-uuid f52fba25-8021-4172-b9c1-88d5728ad329 2026-02-17T21:16:03.104 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:03.093+0000 7f692eb878c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:16:03.105 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:03.097+0000 7f692eb878c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:16:03.106 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:03.097+0000 7f692eb878c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:16:03.107 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:03.097+0000 7f692eb878c0 -1 bdev(0x561bf1bffc00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-17T21:16:03.107 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:03.097+0000 7f692eb878c0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-17T21:16:05.393 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-17T21:16:05.393 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:16:05.394 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T21:16:05.394 INFO:tasks.workunit.client.0.trial031.stdout:adding osd2 key to auth repository 2026-02-17T21:16:05.394 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:16:05.673 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:16:05.674 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T21:16:05.674 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:16:05.674 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:16:05.674 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:16:05.675 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:16:05.700 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:05.693+0000 7f89a99fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:16:05.702 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:05.693+0000 7f89a99fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:16:05.703 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:05.693+0000 7f89a99fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:16:05.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T21:16:05.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:16:05.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:16:05.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:16:05.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:16:05.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:05.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:16:05.870 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:16:05.870 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:05.870 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:16:06.078 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:07.080 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:07.080 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:07.080 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:16:07.081 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:16:07.081 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:07.081 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:16:07.282 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:07.410 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:07.401+0000 7f89a99fc8c0 -1 Falling back to public interface 2026-02-17T21:16:08.284 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:08.284 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:08.284 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:16:08.284 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:16:08.285 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:08.285 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:16:08.492 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:09.493 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:09.493 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:09.494 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:16:09.494 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:16:09.494 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:09.495 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:16:09.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:10.424 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:10.417+0000 7f89a99fc8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T21:16:10.697 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:10.698 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:10.698 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:16:10.699 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:16:10.699 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:10.699 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:16:10.900 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:11.902 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:11.902 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:11.902 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:16:11.903 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:16:11.903 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:11.903 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:16:12.109 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3945420263,v1:127.0.0.1:6819/3945420263] [v2:127.0.0.1:6820/3945420263,v1:127.0.0.1:6821/3945420263] exists,up f52fba25-8021-4172-b9c1-88d5728ad329 2026-02-17T21:16:12.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:16:12.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:16:12.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:16:12.111 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:72: TEST_divergent: ceph osd set noout 2026-02-17T21:16:12.405 INFO:tasks.workunit.client.0.trial031.stderr:noout is set 2026-02-17T21:16:12.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:73: TEST_divergent: ceph osd set noin 2026-02-17T21:16:12.710 INFO:tasks.workunit.client.0.trial031.stderr:noin is set 2026-02-17T21:16:12.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:74: TEST_divergent: ceph osd set nodown 2026-02-17T21:16:13.013 INFO:tasks.workunit.client.0.trial031.stderr:nodown is set 2026-02-17T21:16:13.026 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:75: TEST_divergent: create_pool test 1 1 2026-02-17T21:16:13.026 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T21:16:13.268 INFO:tasks.workunit.client.0.trial031.stderr:pool 'test' already exists 2026-02-17T21:16:13.277 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T21:16:14.278 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:76: TEST_divergent: ceph osd pool set test size 3 2026-02-17T21:16:14.581 INFO:tasks.workunit.client.0.trial031.stderr:set pool 1 size to 3 2026-02-17T21:16:14.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:77: TEST_divergent: ceph osd pool set test min_size 2 2026-02-17T21:16:14.885 INFO:tasks.workunit.client.0.trial031.stderr:set pool 1 min_size to 2 2026-02-17T21:16:14.898 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:79: TEST_divergent: flush_pg_stats 2026-02-17T21:16:14.899 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:16:14.899 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:16:15.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:16:15.100 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:16:15.100 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:16:15.100 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:16:15.100 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:16:15.101 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:16:15.178 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T21:16:15.178 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T21:16:15.178 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T21:16:15.179 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:16:15.179 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:16:15.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-17T21:16:15.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-17T21:16:15.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669' 2026-02-17T21:16:15.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:16:15.259 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:16:15.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574851 2026-02-17T21:16:15.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574851 2026-02-17T21:16:15.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-55834574851' 2026-02-17T21:16:15.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:16:15.339 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T21:16:15.339 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:16:15.340 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:16:15.341 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T21:16:15.341 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:16:15.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T21:16:15.343 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 21474836487 2026-02-17T21:16:15.343 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T21:16:15.343 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:16:15.550 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T21:16:15.550 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:16:15.551 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-17T21:16:15.551 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:16:15.552 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:16:15.553 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-17T21:16:15.553 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:16:15.554 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-17T21:16:15.554 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 38654705669 2026-02-17T21:16:15.555 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-17T21:16:15.555 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:16:15.768 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-17T21:16:15.769 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:16:15.769 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574851 2026-02-17T21:16:15.769 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:16:15.770 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:16:15.771 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574851 2026-02-17T21:16:15.771 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:16:15.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574851 2026-02-17T21:16:15.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-17T21:16:15.773 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 55834574851 2026-02-17T21:16:15.773 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:16:15.977 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-17T21:16:15.977 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:80: TEST_divergent: wait_for_clean 2026-02-17T21:16:15.977 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:16:15.977 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:16:15.977 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:16:15.977 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T21:16:15.978 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:16:15.978 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:16:15.978 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:16:15.978 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:16:15.979 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:16:16.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T21:16:16.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:16:16.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:16:16.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:16:16.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:16:16.072 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:16:16.275 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:16:16.275 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:16:16.276 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:16:16.276 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:16:16.276 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:16:16.276 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:16:16.354 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-17T21:16:16.354 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-17T21:16:16.354 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T21:16:16.355 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:16:16.355 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:16:16.433 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705671 2026-02-17T21:16:16.433 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705671 2026-02-17T21:16:16.433 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705671' 2026-02-17T21:16:16.434 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:16:16.434 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:16:16.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574853 2026-02-17T21:16:16.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574853 2026-02-17T21:16:16.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705671 2-55834574853' 2026-02-17T21:16:16.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:16:16.515 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-17T21:16:16.515 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:16:16.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:16:16.517 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-17T21:16:16.517 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:16:16.518 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-17T21:16:16.518 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 21474836489 2026-02-17T21:16:16.519 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T21:16:16.519 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:16:16.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-17T21:16:16.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:16:17.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:16:17.730 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:16:17.930 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-17T21:16:17.930 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:16:17.931 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705671 2026-02-17T21:16:17.931 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:16:17.932 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:16:17.933 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705671 2026-02-17T21:16:17.933 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:16:17.934 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705671 2026-02-17T21:16:17.934 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-17T21:16:17.934 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 38654705671 2026-02-17T21:16:17.934 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:16:18.140 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705671 -lt 38654705671 2026-02-17T21:16:18.140 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:16:18.141 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574853 2026-02-17T21:16:18.141 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:16:18.142 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:16:18.143 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574853 2026-02-17T21:16:18.143 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:16:18.144 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574853 2026-02-17T21:16:18.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574853' 2026-02-17T21:16:18.145 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 55834574853 2026-02-17T21:16:18.145 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:16:18.347 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574854 -lt 55834574853 2026-02-17T21:16:18.348 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:16:18.348 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:16:18.348 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:16:18.618 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T21:16:18.618 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:16:18.618 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:16:18.618 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:16:18.618 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:16:18.618 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:16:18.619 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:16:18.619 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:16:18.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T21:16:18.822 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:16:18.822 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:16:18.822 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:16:19.089 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T21:16:19.090 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T21:16:19.090 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T21:16:19.090 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:83: TEST_divergent: ceph pg dump pgs --format=json 2026-02-17T21:16:19.091 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:83: TEST_divergent: jq '.pg_stats[0].up_primary' 2026-02-17T21:16:19.286 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:16:19.295 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:83: TEST_divergent: local divergent=1 2026-02-17T21:16:19.296 INFO:tasks.workunit.client.0.trial031.stdout:primary and soon to be divergent is 1 2026-02-17T21:16:19.296 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:84: TEST_divergent: echo 'primary and soon to be divergent is 1' 2026-02-17T21:16:19.296 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:85: TEST_divergent: ceph pg dump pgs 2026-02-17T21:16:19.486 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:16:19.487 INFO:tasks.workunit.client.0.trial031.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T21:16:13.417004+0000 0'0 26:23 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:16:13.211711+0000 0'0 2026-02-17T21:16:13.211711+0000 0 0 periodic scrub scheduled @ 2026-02-19T00:18:48.794255+0000 0 0 2026-02-17T21:16:19.487 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:16:19.487 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:16:19.487 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:16:19.495 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:86: TEST_divergent: local non_divergent= 2026-02-17T21:16:19.495 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:87: TEST_divergent: for i in $osds 2026-02-17T21:16:19.496 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: '[' 0 = 1 ']' 2026-02-17T21:16:19.496 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:92: TEST_divergent: non_divergent=' 0' 2026-02-17T21:16:19.496 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:87: TEST_divergent: for i in $osds 2026-02-17T21:16:19.496 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: '[' 1 = 1 ']' 2026-02-17T21:16:19.496 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:90: TEST_divergent: continue 2026-02-17T21:16:19.496 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:87: TEST_divergent: for i in $osds 2026-02-17T21:16:19.496 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: '[' 2 = 1 ']' 2026-02-17T21:16:19.496 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:92: TEST_divergent: non_divergent=' 0 2' 2026-02-17T21:16:19.497 INFO:tasks.workunit.client.0.trial031.stdout:writing initial objects 2026-02-17T21:16:19.497 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:95: TEST_divergent: echo 'writing initial objects' 2026-02-17T21:16:19.497 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: seq 1 100 2026-02-17T21:16:19.498 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.498 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_1 /etc/fstab 2026-02-17T21:16:19.518 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.519 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_2 /etc/fstab 2026-02-17T21:16:19.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_3 /etc/fstab 2026-02-17T21:16:19.574 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.574 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_4 /etc/fstab 2026-02-17T21:16:19.601 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.602 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_5 /etc/fstab 2026-02-17T21:16:19.629 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.630 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_6 /etc/fstab 2026-02-17T21:16:19.657 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.657 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_7 /etc/fstab 2026-02-17T21:16:19.685 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.685 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_8 /etc/fstab 2026-02-17T21:16:19.713 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.713 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_9 /etc/fstab 2026-02-17T21:16:19.741 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.741 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_10 /etc/fstab 2026-02-17T21:16:19.762 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.762 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_11 /etc/fstab 2026-02-17T21:16:19.788 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.788 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_12 /etc/fstab 2026-02-17T21:16:19.815 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.815 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_13 /etc/fstab 2026-02-17T21:16:19.844 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.844 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_14 /etc/fstab 2026-02-17T21:16:19.873 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.873 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_15 /etc/fstab 2026-02-17T21:16:19.901 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.901 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_16 /etc/fstab 2026-02-17T21:16:19.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_17 /etc/fstab 2026-02-17T21:16:19.957 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.957 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_18 /etc/fstab 2026-02-17T21:16:19.983 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:19.983 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_19 /etc/fstab 2026-02-17T21:16:20.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_20 /etc/fstab 2026-02-17T21:16:20.032 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.032 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_21 /etc/fstab 2026-02-17T21:16:20.059 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.059 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_22 /etc/fstab 2026-02-17T21:16:20.083 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.083 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_23 /etc/fstab 2026-02-17T21:16:20.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_24 /etc/fstab 2026-02-17T21:16:20.137 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.137 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_25 /etc/fstab 2026-02-17T21:16:20.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_26 /etc/fstab 2026-02-17T21:16:20.192 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.193 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_27 /etc/fstab 2026-02-17T21:16:20.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_28 /etc/fstab 2026-02-17T21:16:20.242 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.242 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_29 /etc/fstab 2026-02-17T21:16:20.268 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.269 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_30 /etc/fstab 2026-02-17T21:16:20.295 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.295 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_31 /etc/fstab 2026-02-17T21:16:20.324 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.324 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_32 /etc/fstab 2026-02-17T21:16:20.352 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.352 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_33 /etc/fstab 2026-02-17T21:16:20.380 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.380 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_34 /etc/fstab 2026-02-17T21:16:20.408 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.408 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_35 /etc/fstab 2026-02-17T21:16:20.436 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.436 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_36 /etc/fstab 2026-02-17T21:16:20.464 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.464 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_37 /etc/fstab 2026-02-17T21:16:20.492 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.493 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_38 /etc/fstab 2026-02-17T21:16:20.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_39 /etc/fstab 2026-02-17T21:16:20.548 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.548 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_40 /etc/fstab 2026-02-17T21:16:20.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_41 /etc/fstab 2026-02-17T21:16:20.594 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.595 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_42 /etc/fstab 2026-02-17T21:16:20.621 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.622 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_43 /etc/fstab 2026-02-17T21:16:20.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_44 /etc/fstab 2026-02-17T21:16:20.677 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.677 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_45 /etc/fstab 2026-02-17T21:16:20.705 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.705 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_46 /etc/fstab 2026-02-17T21:16:20.732 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_47 /etc/fstab 2026-02-17T21:16:20.760 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.760 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_48 /etc/fstab 2026-02-17T21:16:20.788 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.789 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_49 /etc/fstab 2026-02-17T21:16:20.816 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.816 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_50 /etc/fstab 2026-02-17T21:16:20.843 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.844 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_51 /etc/fstab 2026-02-17T21:16:20.870 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.870 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_52 /etc/fstab 2026-02-17T21:16:20.897 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.897 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_53 /etc/fstab 2026-02-17T21:16:20.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_54 /etc/fstab 2026-02-17T21:16:20.953 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.953 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_55 /etc/fstab 2026-02-17T21:16:20.981 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:20.981 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_56 /etc/fstab 2026-02-17T21:16:21.007 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.007 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_57 /etc/fstab 2026-02-17T21:16:21.034 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.035 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_58 /etc/fstab 2026-02-17T21:16:21.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_59 /etc/fstab 2026-02-17T21:16:21.089 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.089 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_60 /etc/fstab 2026-02-17T21:16:21.117 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.117 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_61 /etc/fstab 2026-02-17T21:16:21.144 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_62 /etc/fstab 2026-02-17T21:16:21.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_63 /etc/fstab 2026-02-17T21:16:21.192 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.193 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_64 /etc/fstab 2026-02-17T21:16:21.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_65 /etc/fstab 2026-02-17T21:16:21.245 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.245 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_66 /etc/fstab 2026-02-17T21:16:21.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.273 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_67 /etc/fstab 2026-02-17T21:16:21.293 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.293 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_68 /etc/fstab 2026-02-17T21:16:21.319 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.320 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_69 /etc/fstab 2026-02-17T21:16:21.346 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.346 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_70 /etc/fstab 2026-02-17T21:16:21.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_71 /etc/fstab 2026-02-17T21:16:21.402 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.402 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_72 /etc/fstab 2026-02-17T21:16:21.429 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.430 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_73 /etc/fstab 2026-02-17T21:16:21.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_74 /etc/fstab 2026-02-17T21:16:21.485 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.485 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_75 /etc/fstab 2026-02-17T21:16:21.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_76 /etc/fstab 2026-02-17T21:16:21.540 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.540 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_77 /etc/fstab 2026-02-17T21:16:21.566 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.566 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_78 /etc/fstab 2026-02-17T21:16:21.594 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.594 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_79 /etc/fstab 2026-02-17T21:16:21.622 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.622 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_80 /etc/fstab 2026-02-17T21:16:21.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_81 /etc/fstab 2026-02-17T21:16:21.677 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.677 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_82 /etc/fstab 2026-02-17T21:16:21.697 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.697 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_83 /etc/fstab 2026-02-17T21:16:21.726 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.726 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_84 /etc/fstab 2026-02-17T21:16:21.753 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.753 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_85 /etc/fstab 2026-02-17T21:16:21.781 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.781 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_86 /etc/fstab 2026-02-17T21:16:21.808 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.809 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_87 /etc/fstab 2026-02-17T21:16:21.827 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_88 /etc/fstab 2026-02-17T21:16:21.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_89 /etc/fstab 2026-02-17T21:16:21.883 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.883 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_90 /etc/fstab 2026-02-17T21:16:21.912 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.912 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_91 /etc/fstab 2026-02-17T21:16:21.940 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.940 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_92 /etc/fstab 2026-02-17T21:16:21.960 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_93 /etc/fstab 2026-02-17T21:16:21.987 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:21.987 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_94 /etc/fstab 2026-02-17T21:16:22.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:22.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_95 /etc/fstab 2026-02-17T21:16:22.042 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:22.042 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_96 /etc/fstab 2026-02-17T21:16:22.070 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:22.070 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_97 /etc/fstab 2026-02-17T21:16:22.098 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:22.098 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_98 /etc/fstab 2026-02-17T21:16:22.126 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:22.126 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_99 /etc/fstab 2026-02-17T21:16:22.153 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-17T21:16:22.153 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_100 /etc/fstab 2026-02-17T21:16:22.182 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:102: TEST_divergent: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-17T21:16:22.183 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:102: TEST_divergent: wait_for_clean 2026-02-17T21:16:22.183 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:16:22.183 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:16:22.183 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:16:22.184 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-17T21:16:22.184 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:16:22.184 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:16:22.184 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:16:22.184 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:16:22.184 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:16:22.252 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-17T21:16:22.253 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:16:22.253 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:16:22.253 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:16:22.253 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:16:22.253 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:16:22.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:16:22.462 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:16:22.463 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:16:22.463 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:16:22.463 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:16:22.463 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:16:22.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836492 2026-02-17T21:16:22.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836492 2026-02-17T21:16:22.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492' 2026-02-17T21:16:22.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:16:22.546 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:16:22.622 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705674 2026-02-17T21:16:22.623 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705674 2026-02-17T21:16:22.623 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-38654705674' 2026-02-17T21:16:22.623 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:16:22.623 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:16:22.702 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574857 2026-02-17T21:16:22.702 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574857 2026-02-17T21:16:22.702 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-38654705674 2-55834574857' 2026-02-17T21:16:22.703 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:16:22.703 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836492 2026-02-17T21:16:22.703 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:16:22.704 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:16:22.705 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836492 2026-02-17T21:16:22.705 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:16:22.706 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836492 2026-02-17T21:16:22.706 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 21474836492 2026-02-17T21:16:22.707 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-17T21:16:22.707 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:16:22.921 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836492 2026-02-17T21:16:22.921 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:16:23.922 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:16:23.923 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:16:24.123 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836492 2026-02-17T21:16:24.123 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:16:24.124 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705674 2026-02-17T21:16:24.124 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:16:24.125 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:16:24.126 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705674 2026-02-17T21:16:24.126 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:16:24.127 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705674 2026-02-17T21:16:24.128 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 38654705674 2026-02-17T21:16:24.128 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-17T21:16:24.128 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:16:24.346 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705674 2026-02-17T21:16:24.346 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:16:24.347 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574857 2026-02-17T21:16:24.347 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:16:24.348 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:16:24.349 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574857 2026-02-17T21:16:24.349 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:16:24.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574857 2026-02-17T21:16:24.351 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 55834574857 2026-02-17T21:16:24.351 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-17T21:16:24.351 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:16:24.563 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-17T21:16:24.564 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:16:24.564 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:16:24.564 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:16:24.833 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T21:16:24.833 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:16:24.833 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:16:24.833 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:16:24.834 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:16:24.834 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:16:24.834 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:16:24.834 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:16:25.035 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T21:16:25.036 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:16:25.036 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:16:25.037 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:16:25.307 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T21:16:25.307 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T21:16:25.307 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T21:16:25.307 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:104: TEST_divergent: get_pg test existing_1 2026-02-17T21:16:25.307 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-17T21:16:25.307 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=existing_1 2026-02-17T21:16:25.308 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test existing_1 2026-02-17T21:16:25.308 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T21:16:25.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:104: TEST_divergent: local pgid=1.0 2026-02-17T21:16:25.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:107: TEST_divergent: echo 'blackholing osds 0 2' 2026-02-17T21:16:25.514 INFO:tasks.workunit.client.0.trial031.stdout:blackholing osds 0 2 2026-02-17T21:16:25.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: ceph pg dump pgs 2026-02-17T21:16:25.700 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:16:25.700 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 0 0 0 70600 0 0 16 84 16 active+clean 2026-02-17T21:16:13.417004+0000 26'100 26:123 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:16:13.211711+0000 0'0 2026-02-17T21:16:13.211711+0000 0 0 periodic scrub scheduled @ 2026-02-19T00:18:48.794255+0000 0 0 2026-02-17T21:16:25.700 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:16:25.700 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:16:25.701 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:16:25.708 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:109: TEST_divergent: for i in $non_divergent 2026-02-17T21:16:25.709 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: get_asok_path osd.0 2026-02-17T21:16:25.709 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T21:16:25.709 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T21:16:25.709 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:16:25.709 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:16:25.709 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:16:25.710 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.0.asok 2026-02-17T21:16:25.710 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: CEPH_ARGS= 2026-02-17T21:16:25.710 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-17T21:16:25.771 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:16:25.771 INFO:tasks.workunit.client.0.trial031.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-17T21:16:25.771 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:16:25.776 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:109: TEST_divergent: for i in $non_divergent 2026-02-17T21:16:25.777 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: get_asok_path osd.2 2026-02-17T21:16:25.777 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-17T21:16:25.778 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-17T21:16:25.778 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:16:25.778 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:16:25.778 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:16:25.778 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.2.asok 2026-02-17T21:16:25.779 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: CEPH_ARGS= 2026-02-17T21:16:25.779 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-17T21:16:25.827 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:16:25.827 INFO:tasks.workunit.client.0.trial031.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-17T21:16:25.827 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:16:25.833 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:114: TEST_divergent: local case5=100 2026-02-17T21:16:25.833 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:115: TEST_divergent: expr 100 - 1 2026-02-17T21:16:25.834 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:115: TEST_divergent: local case3=99 2026-02-17T21:16:25.834 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:117: TEST_divergent: echo 'writing divergent object' 2026-02-17T21:16:25.834 INFO:tasks.workunit.client.0.trial031.stdout:writing divergent object 2026-02-17T21:16:25.835 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:119: TEST_divergent: echo 'create missing divergent object' 2026-02-17T21:16:25.835 INFO:tasks.workunit.client.0.trial031.stdout:create missing divergent object 2026-02-17T21:16:25.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:120: TEST_divergent: inject_eio rep data test existing_99 td/divergent-priors 0 2026-02-17T21:16:25.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T21:16:25.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T21:16:25.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T21:16:25.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:118: TEST_divergent: rados -p test put existing_100 /etc/fstab 2026-02-17T21:16:25.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T21:16:25.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=test 2026-02-17T21:16:25.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T21:16:25.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=existing_99 2026-02-17T21:16:25.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T21:16:25.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/divergent-priors 2026-02-17T21:16:25.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T21:16:25.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T21:16:25.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T21:16:25.838 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds test existing_99 2026-02-17T21:16:25.838 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=test 2026-02-17T21:16:25.838 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=existing_99 2026-02-17T21:16:25.838 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map test existing_99 2026-02-17T21:16:25.838 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T21:16:26.036 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T21:16:26.036 INFO:tasks.workunit.client.0.trial031.stderr:0 2026-02-17T21:16:26.036 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:16:26.036 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2 2026-02-17T21:16:26.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0' '2') 2026-02-17T21:16:26.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T21:16:26.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-17T21:16:26.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T21:16:26.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T21:16:26.037 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/divergent-priors/1/type 2026-02-17T21:16:26.040 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T21:16:26.040 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-17T21:16:26.040 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T21:16:26.040 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T21:16:26.040 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T21:16:26.040 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T21:16:26.040 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T21:16:26.040 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T21:16:26.041 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T21:16:26.041 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T21:16:26.041 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:16:26.041 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:16:26.041 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:16:26.041 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.1.asok 2026-02-17T21:16:26.041 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.16076/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-17T21:16:26.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T21:16:26.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T21:16:26.101 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T21:16:26.101 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-17T21:16:26.101 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T21:16:26.101 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T21:16:26.101 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:16:26.101 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:16:26.101 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:16:26.101 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.1.asok 2026-02-17T21:16:26.101 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T21:16:26.101 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.1.asok injectdataerr test existing_99 2026-02-17T21:16:26.159 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:122: TEST_divergent: sleep 10 2026-02-17T21:16:26.160 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:121: TEST_divergent: rados -p test get existing_99 td/divergent-priors/existing 2026-02-17T21:16:36.160 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:123: TEST_divergent: killall -9 rados 2026-02-17T21:16:36.206 INFO:tasks.workunit.client.0.trial031.stderr:rados: no process found 2026-02-17T21:16:36.206 INFO:tasks.workunit.client.0.trial031.stdout:killing all the osds 2026-02-17T21:16:36.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:126: TEST_divergent: echo 'killing all the osds' 2026-02-17T21:16:36.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:127: TEST_divergent: ceph pg dump pgs 2026-02-17T21:16:36.410 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:16:36.410 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 0 0 0 70600 0 0 17 84 17 active+clean 2026-02-17T21:16:26.177127+0000 26'101 26:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:16:13.211711+0000 0'0 2026-02-17T21:16:13.211711+0000 0 0 periodic scrub scheduled @ 2026-02-19T00:18:48.794255+0000 0 0 2026-02-17T21:16:36.411 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:16:36.411 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:16:36.411 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:16:36.420 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:128: TEST_divergent: kill_daemons td/divergent-priors KILL osd 2026-02-17T21:16:36.421 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:16:36.421 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:16:36.422 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:16:36.422 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:16:36.422 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:16:36.530 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:16:36.530 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:129: TEST_divergent: for i in $osds 2026-02-17T21:16:36.530 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:131: TEST_divergent: ceph osd down osd.0 2026-02-17T21:16:36.786 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already down. 2026-02-17T21:16:36.796 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:129: TEST_divergent: for i in $osds 2026-02-17T21:16:36.796 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:131: TEST_divergent: ceph osd down osd.1 2026-02-17T21:16:37.049 INFO:tasks.workunit.client.0.trial031.stderr:osd.1 is already down. 2026-02-17T21:16:37.059 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:129: TEST_divergent: for i in $osds 2026-02-17T21:16:37.059 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:131: TEST_divergent: ceph osd down osd.2 2026-02-17T21:16:37.356 INFO:tasks.workunit.client.0.trial031.stderr:osd.2 is already down. 2026-02-17T21:16:37.365 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:133: TEST_divergent: for i in $non_divergent 2026-02-17T21:16:37.366 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:135: TEST_divergent: ceph osd out osd.0 2026-02-17T21:16:37.658 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already out. 2026-02-17T21:16:37.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:133: TEST_divergent: for i in $non_divergent 2026-02-17T21:16:37.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:135: TEST_divergent: ceph osd out osd.2 2026-02-17T21:16:37.936 INFO:tasks.workunit.client.0.trial031.stderr:osd.2 is already out. 2026-02-17T21:16:37.945 INFO:tasks.workunit.client.0.trial031.stdout:bringing up non_divergent 0 2 2026-02-17T21:16:37.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:139: TEST_divergent: echo 'bringing up non_divergent 0 2' 2026-02-17T21:16:37.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:140: TEST_divergent: ceph pg dump pgs 2026-02-17T21:16:38.152 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:16:38.152 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 0 0 0 70600 0 0 17 84 17 stale+active+clean 2026-02-17T21:16:26.177127+0000 26'101 26:132 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:16:13.211711+0000 0'0 2026-02-17T21:16:13.211711+0000 0 0 periodic scrub scheduled @ 2026-02-19T00:18:48.794255+0000 0 0 2026-02-17T21:16:38.152 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:16:38.152 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:16:38.153 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:16:38.163 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:141: TEST_divergent: for i in $non_divergent 2026-02-17T21:16:38.163 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:143: TEST_divergent: activate_osd td/divergent-priors 0 2026-02-17T21:16:38.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:16:38.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:16:38.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T21:16:38.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:16:38.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/0 2026-02-17T21:16:38.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:16:38.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:16:38.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:16:38.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:16:38.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-17T21:16:38.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-17T21:16:38.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:16:38.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:16:38.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:16:38.165 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:16:38.165 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:16:38.165 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:16:38.165 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:16:38.165 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:16:38.165 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:16:38.165 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:16:38.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:16:38.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:16:38.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:16:38.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:16:38.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:16:38.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:16:38.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:16:38.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:16:38.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:16:38.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:16:38.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/0 2026-02-17T21:16:38.167 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:16:38.167 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T21:16:38.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:16:38.168 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/0/whoami 2026-02-17T21:16:38.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T21:16:38.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:16:38.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:16:38.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:16:38.181 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:38.172+0000 7f051b1cd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:16:38.183 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:38.172+0000 7f051b1cd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:16:38.183 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:38.172+0000 7f051b1cd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:16:38.368 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T21:16:38.368 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:16:38.368 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:16:38.368 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:16:38.368 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:16:38.368 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:38.368 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:16:38.369 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:16:38.369 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:38.369 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:16:38.575 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:39.430 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:39.420+0000 7f051b1cd8c0 -1 Falling back to public interface 2026-02-17T21:16:39.576 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:39.576 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:39.576 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:16:39.577 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:16:39.577 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:39.577 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:16:39.779 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:40.780 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:40.780 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:40.781 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:16:40.781 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:16:40.781 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:40.782 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:16:40.989 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:41.991 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:41.991 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:41.991 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:16:41.991 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:16:41.992 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:41.992 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:16:42.193 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:42.703 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:42.692+0000 7f051b1cd8c0 -1 osd.0 26 log_to_monitors true 2026-02-17T21:16:43.194 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:43.194 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:43.195 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:16:43.195 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:16:43.196 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:43.196 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:16:43.397 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up out weight 0 up_from 33 up_thru 0 down_at 27 last_clean_interval [5,26) [v2:127.0.0.1:6802/507594524,v1:127.0.0.1:6803/507594524] [v2:127.0.0.1:6804/507594524,v1:127.0.0.1:6805/507594524] exists,up f21d462f-1bd6-4e5f-afc1-346559015142 2026-02-17T21:16:43.400 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:16:43.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:16:43.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:16:43.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:141: TEST_divergent: for i in $non_divergent 2026-02-17T21:16:43.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:143: TEST_divergent: activate_osd td/divergent-priors 2 2026-02-17T21:16:43.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:16:43.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:16:43.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T21:16:43.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:16:43.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/2 2026-02-17T21:16:43.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:16:43.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:16:43.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:16:43.402 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:16:43.402 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-17T21:16:43.402 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-17T21:16:43.402 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:16:43.402 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:16:43.402 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:16:43.402 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:16:43.402 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:16:43.402 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:16:43.402 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:16:43.402 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:16:43.402 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:16:43.402 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:16:43.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:16:43.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:16:43.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:16:43.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:16:43.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:16:43.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:16:43.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:16:43.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:16:43.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:16:43.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:16:43.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/2 2026-02-17T21:16:43.404 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:16:43.405 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T21:16:43.405 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:16:43.405 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/2/whoami 2026-02-17T21:16:43.405 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T21:16:43.405 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:16:43.405 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:16:43.405 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:16:43.422 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:43.412+0000 7fd6f53278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:16:43.424 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:43.412+0000 7fd6f53278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:16:43.425 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:43.416+0000 7fd6f53278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:16:43.602 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T21:16:43.602 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:16:43.602 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:16:43.602 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:16:43.603 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:16:43.603 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:43.603 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:16:43.604 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:16:43.604 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:43.604 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:16:43.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:44.646 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:44.636+0000 7fd6f53278c0 -1 Falling back to public interface 2026-02-17T21:16:44.813 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:44.813 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:44.813 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:16:44.814 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:16:44.814 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:44.814 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:16:45.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:46.015 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:46.015 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:46.016 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:16:46.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:16:46.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:46.017 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:16:46.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:47.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:47.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:47.220 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:16:47.221 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:16:47.221 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:47.221 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:16:47.438 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:16:47.685 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:16:47.676+0000 7fd6f53278c0 -1 osd.2 26 log_to_monitors true 2026-02-17T21:16:48.439 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:16:48.439 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:16:48.440 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:16:48.441 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:16:48.441 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:16:48.441 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:16:48.643 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up out weight 0 up_from 35 up_thru 0 down_at 29 last_clean_interval [13,26) [v2:127.0.0.1:6810/850722593,v1:127.0.0.1:6811/850722593] [v2:127.0.0.1:6812/850722593,v1:127.0.0.1:6813/850722593] exists,up f52fba25-8021-4172-b9c1-88d5728ad329 2026-02-17T21:16:48.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:16:48.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:16:48.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:16:48.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:145: TEST_divergent: for i in $non_divergent 2026-02-17T21:16:48.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:147: TEST_divergent: ceph osd in osd.0 2026-02-17T21:16:48.897 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already in. 2026-02-17T21:16:48.906 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:145: TEST_divergent: for i in $non_divergent 2026-02-17T21:16:48.906 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:147: TEST_divergent: ceph osd in osd.2 2026-02-17T21:16:49.202 INFO:tasks.workunit.client.0.trial031.stderr:osd.2 is already in. 2026-02-17T21:16:49.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:150: TEST_divergent: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-17T21:16:49.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:150: TEST_divergent: wait_for_clean 2026-02-17T21:16:49.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:16:49.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:16:49.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:16:49.212 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-17T21:16:49.212 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:16:49.212 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:16:49.213 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:16:49.213 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:16:49.213 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:16:49.268 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-17T21:16:49.268 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:16:49.269 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:16:49.269 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:16:49.269 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:16:49.269 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:16:49.470 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:16:49.471 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:16:49.471 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:16:49.471 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:16:49.471 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:16:49.471 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:16:49.550 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=141733920772 2026-02-17T21:16:49.550 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 141733920772 2026-02-17T21:16:49.550 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-141733920772' 2026-02-17T21:16:49.550 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:16:49.550 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:16:49.610 INFO:tasks.workunit.client.0.trial031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-17T21:16:49.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-17T21:16:49.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-17T21:16:49.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-17T21:16:49.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:16:49.613 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:16:49.689 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855363 2026-02-17T21:16:49.690 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855363 2026-02-17T21:16:49.690 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-141733920772 2-150323855363' 2026-02-17T21:16:49.690 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:16:49.690 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-141733920772 2026-02-17T21:16:49.690 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:16:49.691 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:16:49.691 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-141733920772 2026-02-17T21:16:49.691 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:16:49.693 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=141733920772 2026-02-17T21:16:49.693 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 141733920772 2026-02-17T21:16:49.693 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 141733920772' 2026-02-17T21:16:49.694 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:16:49.899 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920770 -lt 141733920772 2026-02-17T21:16:49.899 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:16:50.900 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:16:50.901 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:16:51.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920770 -lt 141733920772 2026-02-17T21:16:51.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:16:52.100 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T21:16:52.100 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:16:52.304 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920772 -lt 141733920772 2026-02-17T21:16:52.304 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:16:52.305 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-150323855363 2026-02-17T21:16:52.305 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:16:52.306 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:16:52.307 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-150323855363 2026-02-17T21:16:52.307 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:16:52.308 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855363 2026-02-17T21:16:52.308 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 150323855363 2026-02-17T21:16:52.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 150323855363' 2026-02-17T21:16:52.309 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:16:52.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855363 -lt 150323855363 2026-02-17T21:16:52.516 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:16:52.516 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:16:52.516 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:16:52.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T21:16:52.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:16:52.774 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:16:52.774 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:16:52.774 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:16:52.775 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:16:52.775 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:16:52.775 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:16:52.968 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:16:52.968 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:16:52.968 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:16:52.968 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:16:53.228 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:16:53.228 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T21:16:53.229 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T21:16:53.229 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T21:16:53.229 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:16:53.229 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T21:16:53.329 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:16:53.330 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:16:53.330 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:16:53.330 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:16:53.330 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:16:53.330 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:16:53.331 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:16:53.331 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:16:53.527 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:16:53.527 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:16:53.528 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:16:53.528 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:16:53.786 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:16:53.786 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:16:53.786 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:16:53.786 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:16:53.786 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:16:53.786 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:16:53.786 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:16:53.787 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:16:53.787 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:16:54.050 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:16:54.050 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:16:54.051 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-17T21:16:54.051 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:16:54.051 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T21:16:54.251 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:16:54.252 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:16:54.252 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:16:54.252 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:16:54.252 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:16:54.252 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:16:54.253 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:16:54.253 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:16:54.448 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:16:54.449 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:16:54.449 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:16:54.449 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:16:54.705 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:16:54.705 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:16:54.706 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:16:54.706 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:16:54.706 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:16:54.706 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:16:54.706 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:16:54.706 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:16:54.706 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:16:54.963 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:16:54.963 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:16:54.964 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-17T21:16:54.964 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:16:54.964 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T21:16:55.364 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:16:55.364 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:16:55.364 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:16:55.364 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:16:55.364 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:16:55.364 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:16:55.365 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:16:55.365 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:16:55.563 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:16:55.563 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:16:55.564 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:16:55.564 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:16:55.822 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:16:55.822 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:16:55.822 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:16:55.822 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:16:55.822 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:16:55.822 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:16:55.822 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:16:55.822 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:16:55.823 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:16:56.085 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:16:56.086 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:16:56.086 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-17T21:16:56.086 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:16:56.086 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T21:16:56.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:16:56.887 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:16:56.888 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:16:56.888 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:16:56.888 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:16:56.888 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:16:56.888 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:16:56.888 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:16:57.084 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:16:57.085 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:16:57.085 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:16:57.085 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:16:57.347 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:16:57.348 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:16:57.348 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:16:57.348 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:16:57.348 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:16:57.348 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:16:57.348 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:16:57.348 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:16:57.348 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:16:57.615 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:16:57.615 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:16:57.616 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-17T21:16:57.616 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:16:57.616 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-17T21:16:59.216 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:16:59.216 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:16:59.217 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:16:59.217 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:16:59.217 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:16:59.217 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:16:59.218 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:16:59.218 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:16:59.421 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:16:59.422 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:16:59.422 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:16:59.422 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:16:59.697 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:16:59.697 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:16:59.697 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:16:59.697 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:16:59.697 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:16:59.697 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:16:59.697 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:16:59.698 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:16:59.698 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:16:59.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:16:59.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:16:59.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-17T21:16:59.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:16:59.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-17T21:17:03.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:17:03.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:17:03.169 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:17:03.169 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:17:03.169 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:17:03.169 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:17:03.169 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:17:03.170 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:17:03.374 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:17:03.374 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:17:03.374 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:17:03.375 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:17:03.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:17:03.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:17:03.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:17:03.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:17:03.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:17:03.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:17:03.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:17:03.645 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:17:03.645 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:17:03.908 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:17:03.908 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:17:03.909 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-17T21:17:03.909 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:17:03.909 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-17T21:17:10.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:17:10.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:17:10.310 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:17:10.311 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:17:10.311 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:17:10.311 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:17:10.311 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:17:10.311 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:17:10.507 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:17:10.507 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:17:10.507 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:17:10.508 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:17:10.771 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:17:10.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:17:10.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:17:10.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:17:10.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:17:10.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:17:10.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:17:10.772 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:17:10.773 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:17:11.028 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:17:11.029 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:17:11.029 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-17T21:17:11.029 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:17:11.029 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-17T21:17:18.330 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:17:18.331 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:17:18.331 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:17:18.331 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:17:18.331 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:17:18.332 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:17:18.332 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:17:18.332 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:17:18.540 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:17:18.541 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:17:18.541 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:17:18.541 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:17:18.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:17:18.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:17:18.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:17:18.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:17:18.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:17:18.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:17:18.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:17:18.811 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:17:18.811 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:17:19.079 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:17:19.079 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:17:19.079 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-17T21:17:19.079 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-17T21:17:19.341 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:17:19.341 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_fingerprint": "eb4be5b2-876b-439c-a43c-8bc96b59fd83", 2026-02-17T21:17:19.341 INFO:tasks.workunit.client.0.trial031.stdout: "version": "19.2.3-679-gd855f53b", 2026-02-17T21:17:19.342 INFO:tasks.workunit.client.0.trial031.stdout: "commit": "d855f53b89fdcec760fd9232a5fb55ed4fb111a1", 2026-02-17T21:17:19.342 INFO:tasks.workunit.client.0.trial031.stdout: "timestamp": "2026-02-17T21:17:19.336754+0000", 2026-02-17T21:17:19.342 INFO:tasks.workunit.client.0.trial031.stdout: "tag": "", 2026-02-17T21:17:19.342 INFO:tasks.workunit.client.0.trial031.stdout: "health": { 2026-02-17T21:17:19.342 INFO:tasks.workunit.client.0.trial031.stdout: "status": "HEALTH_WARN", 2026-02-17T21:17:19.342 INFO:tasks.workunit.client.0.trial031.stdout: "checks": { 2026-02-17T21:17:19.342 INFO:tasks.workunit.client.0.trial031.stdout: "OSDMAP_FLAGS": { 2026-02-17T21:17:19.342 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:17:19.342 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:17:19.342 INFO:tasks.workunit.client.0.trial031.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-17T21:17:19.342 INFO:tasks.workunit.client.0.trial031.stdout: "count": 17 2026-02-17T21:17:19.342 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.343 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [], 2026-02-17T21:17:19.343 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:17:19.343 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.343 INFO:tasks.workunit.client.0.trial031.stdout: "OSD_DOWN": { 2026-02-17T21:17:19.343 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:17:19.343 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:17:19.343 INFO:tasks.workunit.client.0.trial031.stdout: "message": "1 osds down", 2026-02-17T21:17:19.343 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:17:19.343 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.343 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:17:19.343 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.343 INFO:tasks.workunit.client.0.trial031.stdout: "message": "osd.1 (root=default,host=trial031) is down" 2026-02-17T21:17:19.344 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.344 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.344 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:17:19.344 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.344 INFO:tasks.workunit.client.0.trial031.stdout: "PG_DEGRADED": { 2026-02-17T21:17:19.344 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:17:19.344 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:17:19.344 INFO:tasks.workunit.client.0.trial031.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2026-02-17T21:17:19.344 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:17:19.344 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.344 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:17:19.344 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.345 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:17:19.345 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.345 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.345 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:17:19.345 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.345 INFO:tasks.workunit.client.0.trial031.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-17T21:17:19.345 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:17:19.345 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:17:19.345 INFO:tasks.workunit.client.0.trial031.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-17T21:17:19.345 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:17:19.345 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.345 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:17:19.345 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.346 INFO:tasks.workunit.client.0.trial031.stdout: "message": "application not enabled on pool 'test'" 2026-02-17T21:17:19.346 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.346 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.346 INFO:tasks.workunit.client.0.trial031.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-17T21:17:19.346 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.346 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.346 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:17:19.346 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.346 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.346 INFO:tasks.workunit.client.0.trial031.stdout: "mutes": [] 2026-02-17T21:17:19.346 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.346 INFO:tasks.workunit.client.0.trial031.stdout: "monmap_first_committed": 1, 2026-02-17T21:17:19.346 INFO:tasks.workunit.client.0.trial031.stdout: "monmap_last_committed": 1, 2026-02-17T21:17:19.347 INFO:tasks.workunit.client.0.trial031.stdout: "monmap": { 2026-02-17T21:17:19.347 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 1, 2026-02-17T21:17:19.347 INFO:tasks.workunit.client.0.trial031.stdout: "fsid": "5ebac6e7-5321-4893-ad68-c06127ed455b", 2026-02-17T21:17:19.347 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:15:44.888124Z", 2026-02-17T21:17:19.347 INFO:tasks.workunit.client.0.trial031.stdout: "created": "2026-02-17T21:15:44.888124Z", 2026-02-17T21:17:19.347 INFO:tasks.workunit.client.0.trial031.stdout: "min_mon_release": 19, 2026-02-17T21:17:19.347 INFO:tasks.workunit.client.0.trial031.stdout: "min_mon_release_name": "squid", 2026-02-17T21:17:19.347 INFO:tasks.workunit.client.0.trial031.stdout: "election_strategy": 1, 2026-02-17T21:17:19.347 INFO:tasks.workunit.client.0.trial031.stdout: "disallowed_leaders": "", 2026-02-17T21:17:19.347 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode": false, 2026-02-17T21:17:19.347 INFO:tasks.workunit.client.0.trial031.stdout: "tiebreaker_mon": "", 2026-02-17T21:17:19.347 INFO:tasks.workunit.client.0.trial031.stdout: "removed_ranks": "", 2026-02-17T21:17:19.347 INFO:tasks.workunit.client.0.trial031.stdout: "features": { 2026-02-17T21:17:19.348 INFO:tasks.workunit.client.0.trial031.stdout: "persistent": [ 2026-02-17T21:17:19.348 INFO:tasks.workunit.client.0.trial031.stdout: "kraken", 2026-02-17T21:17:19.348 INFO:tasks.workunit.client.0.trial031.stdout: "luminous", 2026-02-17T21:17:19.348 INFO:tasks.workunit.client.0.trial031.stdout: "mimic", 2026-02-17T21:17:19.348 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap-prune", 2026-02-17T21:17:19.348 INFO:tasks.workunit.client.0.trial031.stdout: "nautilus", 2026-02-17T21:17:19.348 INFO:tasks.workunit.client.0.trial031.stdout: "octopus", 2026-02-17T21:17:19.348 INFO:tasks.workunit.client.0.trial031.stdout: "pacific", 2026-02-17T21:17:19.348 INFO:tasks.workunit.client.0.trial031.stdout: "elector-pinging", 2026-02-17T21:17:19.348 INFO:tasks.workunit.client.0.trial031.stdout: "quincy", 2026-02-17T21:17:19.348 INFO:tasks.workunit.client.0.trial031.stdout: "reef", 2026-02-17T21:17:19.348 INFO:tasks.workunit.client.0.trial031.stdout: "squid" 2026-02-17T21:17:19.348 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.349 INFO:tasks.workunit.client.0.trial031.stdout: "optional": [] 2026-02-17T21:17:19.349 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.349 INFO:tasks.workunit.client.0.trial031.stdout: "mons": [ 2026-02-17T21:17:19.349 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.349 INFO:tasks.workunit.client.0.trial031.stdout: "rank": 0, 2026-02-17T21:17:19.349 INFO:tasks.workunit.client.0.trial031.stdout: "name": "a", 2026-02-17T21:17:19.349 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:17:19.349 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:17:19.349 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.349 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:17:19.349 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:7115", 2026-02-17T21:17:19.349 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 0 2026-02-17T21:17:19.349 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.350 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.350 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.350 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:7115/0", 2026-02-17T21:17:19.350 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-17T21:17:19.350 INFO:tasks.workunit.client.0.trial031.stdout: "priority": 0, 2026-02-17T21:17:19.350 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 0, 2026-02-17T21:17:19.350 INFO:tasks.workunit.client.0.trial031.stdout: "crush_location": "{}" 2026-02-17T21:17:19.350 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.350 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.350 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.350 INFO:tasks.workunit.client.0.trial031.stdout: "quorum": [ 2026-02-17T21:17:19.350 INFO:tasks.workunit.client.0.trial031.stdout: 0 2026-02-17T21:17:19.350 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.351 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap": { 2026-02-17T21:17:19.351 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 40, 2026-02-17T21:17:19.351 INFO:tasks.workunit.client.0.trial031.stdout: "fsid": "5ebac6e7-5321-4893-ad68-c06127ed455b", 2026-02-17T21:17:19.351 INFO:tasks.workunit.client.0.trial031.stdout: "created": "2026-02-17T21:15:44.973350+0000", 2026-02-17T21:17:19.351 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:16:49.248202+0000", 2026-02-17T21:17:19.351 INFO:tasks.workunit.client.0.trial031.stdout: "last_up_change": "2026-02-17T21:16:47.835640+0000", 2026-02-17T21:17:19.351 INFO:tasks.workunit.client.0.trial031.stdout: "last_in_change": "2026-02-17T21:16:49.146410+0000", 2026-02-17T21:17:19.351 INFO:tasks.workunit.client.0.trial031.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-17T21:17:19.351 INFO:tasks.workunit.client.0.trial031.stdout: "flags_num": 5800384, 2026-02-17T21:17:19.351 INFO:tasks.workunit.client.0.trial031.stdout: "flags_set": [ 2026-02-17T21:17:19.351 INFO:tasks.workunit.client.0.trial031.stdout: "nodown", 2026-02-17T21:17:19.351 INFO:tasks.workunit.client.0.trial031.stdout: "noin", 2026-02-17T21:17:19.351 INFO:tasks.workunit.client.0.trial031.stdout: "noout", 2026-02-17T21:17:19.351 INFO:tasks.workunit.client.0.trial031.stdout: "pglog_hardlimit", 2026-02-17T21:17:19.352 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snapdirs", 2026-02-17T21:17:19.352 INFO:tasks.workunit.client.0.trial031.stdout: "recovery_deletes", 2026-02-17T21:17:19.352 INFO:tasks.workunit.client.0.trial031.stdout: "sortbitwise" 2026-02-17T21:17:19.352 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.352 INFO:tasks.workunit.client.0.trial031.stdout: "crush_version": 9, 2026-02-17T21:17:19.352 INFO:tasks.workunit.client.0.trial031.stdout: "full_ratio": 0.99000000953674316, 2026-02-17T21:17:19.352 INFO:tasks.workunit.client.0.trial031.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-17T21:17:19.352 INFO:tasks.workunit.client.0.trial031.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-17T21:17:19.352 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_snapshot": "", 2026-02-17T21:17:19.352 INFO:tasks.workunit.client.0.trial031.stdout: "pool_max": 1, 2026-02-17T21:17:19.352 INFO:tasks.workunit.client.0.trial031.stdout: "max_osd": 3, 2026-02-17T21:17:19.352 INFO:tasks.workunit.client.0.trial031.stdout: "require_min_compat_client": "luminous", 2026-02-17T21:17:19.352 INFO:tasks.workunit.client.0.trial031.stdout: "min_compat_client": "jewel", 2026-02-17T21:17:19.353 INFO:tasks.workunit.client.0.trial031.stdout: "require_osd_release": "squid", 2026-02-17T21:17:19.353 INFO:tasks.workunit.client.0.trial031.stdout: "allow_crimson": false, 2026-02-17T21:17:19.353 INFO:tasks.workunit.client.0.trial031.stdout: "pools": [ 2026-02-17T21:17:19.353 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.353 INFO:tasks.workunit.client.0.trial031.stdout: "pool": 1, 2026-02-17T21:17:19.353 INFO:tasks.workunit.client.0.trial031.stdout: "pool_name": "test", 2026-02-17T21:17:19.353 INFO:tasks.workunit.client.0.trial031.stdout: "create_time": "2026-02-17T21:16:13.175507+0000", 2026-02-17T21:17:19.353 INFO:tasks.workunit.client.0.trial031.stdout: "flags": 1, 2026-02-17T21:17:19.353 INFO:tasks.workunit.client.0.trial031.stdout: "flags_names": "hashpspool", 2026-02-17T21:17:19.353 INFO:tasks.workunit.client.0.trial031.stdout: "type": 1, 2026-02-17T21:17:19.353 INFO:tasks.workunit.client.0.trial031.stdout: "size": 3, 2026-02-17T21:17:19.353 INFO:tasks.workunit.client.0.trial031.stdout: "min_size": 2, 2026-02-17T21:17:19.354 INFO:tasks.workunit.client.0.trial031.stdout: "crush_rule": 0, 2026-02-17T21:17:19.354 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_count": 0, 2026-02-17T21:17:19.354 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_target": 0, 2026-02-17T21:17:19.354 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_barrier": 0, 2026-02-17T21:17:19.354 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-17T21:17:19.354 INFO:tasks.workunit.client.0.trial031.stdout: "is_stretch_pool": false, 2026-02-17T21:17:19.354 INFO:tasks.workunit.client.0.trial031.stdout: "object_hash": 2, 2026-02-17T21:17:19.354 INFO:tasks.workunit.client.0.trial031.stdout: "pg_autoscale_mode": "off", 2026-02-17T21:17:19.354 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num": 1, 2026-02-17T21:17:19.354 INFO:tasks.workunit.client.0.trial031.stdout: "pg_placement_num": 1, 2026-02-17T21:17:19.354 INFO:tasks.workunit.client.0.trial031.stdout: "pg_placement_num_target": 1, 2026-02-17T21:17:19.354 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num_target": 1, 2026-02-17T21:17:19.354 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num_pending": 1, 2026-02-17T21:17:19.354 INFO:tasks.workunit.client.0.trial031.stdout: "last_pg_merge_meta": { 2026-02-17T21:17:19.355 INFO:tasks.workunit.client.0.trial031.stdout: "source_pgid": "0.0", 2026-02-17T21:17:19.355 INFO:tasks.workunit.client.0.trial031.stdout: "ready_epoch": 0, 2026-02-17T21:17:19.355 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_started": 0, 2026-02-17T21:17:19.355 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_clean": 0, 2026-02-17T21:17:19.355 INFO:tasks.workunit.client.0.trial031.stdout: "source_version": "0'0", 2026-02-17T21:17:19.355 INFO:tasks.workunit.client.0.trial031.stdout: "target_version": "0'0" 2026-02-17T21:17:19.355 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.355 INFO:tasks.workunit.client.0.trial031.stdout: "last_change": "26", 2026-02-17T21:17:19.355 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend": "0", 2026-02-17T21:17:19.355 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-17T21:17:19.355 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend_preluminous": "0", 2026-02-17T21:17:19.355 INFO:tasks.workunit.client.0.trial031.stdout: "auid": 0, 2026-02-17T21:17:19.355 INFO:tasks.workunit.client.0.trial031.stdout: "snap_mode": "selfmanaged", 2026-02-17T21:17:19.356 INFO:tasks.workunit.client.0.trial031.stdout: "snap_seq": 0, 2026-02-17T21:17:19.356 INFO:tasks.workunit.client.0.trial031.stdout: "snap_epoch": 0, 2026-02-17T21:17:19.356 INFO:tasks.workunit.client.0.trial031.stdout: "pool_snaps": [], 2026-02-17T21:17:19.356 INFO:tasks.workunit.client.0.trial031.stdout: "removed_snaps": "[]", 2026-02-17T21:17:19.356 INFO:tasks.workunit.client.0.trial031.stdout: "quota_max_bytes": 0, 2026-02-17T21:17:19.356 INFO:tasks.workunit.client.0.trial031.stdout: "quota_max_objects": 0, 2026-02-17T21:17:19.356 INFO:tasks.workunit.client.0.trial031.stdout: "tiers": [], 2026-02-17T21:17:19.356 INFO:tasks.workunit.client.0.trial031.stdout: "tier_of": -1, 2026-02-17T21:17:19.356 INFO:tasks.workunit.client.0.trial031.stdout: "read_tier": -1, 2026-02-17T21:17:19.356 INFO:tasks.workunit.client.0.trial031.stdout: "write_tier": -1, 2026-02-17T21:17:19.356 INFO:tasks.workunit.client.0.trial031.stdout: "cache_mode": "none", 2026-02-17T21:17:19.356 INFO:tasks.workunit.client.0.trial031.stdout: "target_max_bytes": 0, 2026-02-17T21:17:19.356 INFO:tasks.workunit.client.0.trial031.stdout: "target_max_objects": 0, 2026-02-17T21:17:19.356 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-17T21:17:19.357 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-17T21:17:19.357 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-17T21:17:19.357 INFO:tasks.workunit.client.0.trial031.stdout: "cache_min_flush_age": 0, 2026-02-17T21:17:19.357 INFO:tasks.workunit.client.0.trial031.stdout: "cache_min_evict_age": 0, 2026-02-17T21:17:19.357 INFO:tasks.workunit.client.0.trial031.stdout: "erasure_code_profile": "", 2026-02-17T21:17:19.357 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_params": { 2026-02-17T21:17:19.357 INFO:tasks.workunit.client.0.trial031.stdout: "type": "none" 2026-02-17T21:17:19.357 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.357 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_period": 0, 2026-02-17T21:17:19.357 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_count": 0, 2026-02-17T21:17:19.357 INFO:tasks.workunit.client.0.trial031.stdout: "use_gmt_hitset": true, 2026-02-17T21:17:19.357 INFO:tasks.workunit.client.0.trial031.stdout: "min_read_recency_for_promote": 0, 2026-02-17T21:17:19.357 INFO:tasks.workunit.client.0.trial031.stdout: "min_write_recency_for_promote": 0, 2026-02-17T21:17:19.358 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_grade_decay_rate": 0, 2026-02-17T21:17:19.358 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_search_last_n": 0, 2026-02-17T21:17:19.358 INFO:tasks.workunit.client.0.trial031.stdout: "grade_table": [], 2026-02-17T21:17:19.358 INFO:tasks.workunit.client.0.trial031.stdout: "stripe_width": 0, 2026-02-17T21:17:19.358 INFO:tasks.workunit.client.0.trial031.stdout: "expected_num_objects": 0, 2026-02-17T21:17:19.358 INFO:tasks.workunit.client.0.trial031.stdout: "fast_read": false, 2026-02-17T21:17:19.358 INFO:tasks.workunit.client.0.trial031.stdout: "options": {}, 2026-02-17T21:17:19.358 INFO:tasks.workunit.client.0.trial031.stdout: "application_metadata": {}, 2026-02-17T21:17:19.358 INFO:tasks.workunit.client.0.trial031.stdout: "read_balance": { 2026-02-17T21:17:19.358 INFO:tasks.workunit.client.0.trial031.stdout: "score_type": "Fair distribution", 2026-02-17T21:17:19.358 INFO:tasks.workunit.client.0.trial031.stdout: "score_acting": 2.9900000095367432, 2026-02-17T21:17:19.358 INFO:tasks.workunit.client.0.trial031.stdout: "score_stable": 2.9900000095367432, 2026-02-17T21:17:19.358 INFO:tasks.workunit.client.0.trial031.stdout: "optimal_score": 0.67000001668930054, 2026-02-17T21:17:19.358 INFO:tasks.workunit.client.0.trial031.stdout: "raw_score_acting": 2, 2026-02-17T21:17:19.359 INFO:tasks.workunit.client.0.trial031.stdout: "raw_score_stable": 2, 2026-02-17T21:17:19.359 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity_weighted": 1, 2026-02-17T21:17:19.359 INFO:tasks.workunit.client.0.trial031.stdout: "average_primary_affinity": 1, 2026-02-17T21:17:19.359 INFO:tasks.workunit.client.0.trial031.stdout: "average_primary_affinity_weighted": 1 2026-02-17T21:17:19.359 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.359 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.359 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.359 INFO:tasks.workunit.client.0.trial031.stdout: "osds": [ 2026-02-17T21:17:19.359 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.359 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:17:19.359 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "f21d462f-1bd6-4e5f-afc1-346559015142", 2026-02-17T21:17:19.359 INFO:tasks.workunit.client.0.trial031.stdout: "up": 1, 2026-02-17T21:17:19.359 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:17:19.360 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:17:19.360 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:17:19.360 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 5, 2026-02-17T21:17:19.360 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 26, 2026-02-17T21:17:19.360 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 33, 2026-02-17T21:17:19.360 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 39, 2026-02-17T21:17:19.360 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 27, 2026-02-17T21:17:19.360 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:17:19.360 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:17:19.360 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:17:19.360 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.360 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:17:19.360 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6802", 2026-02-17T21:17:19.361 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 507594524 2026-02-17T21:17:19.361 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.361 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.361 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:17:19.361 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6803", 2026-02-17T21:17:19.361 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 507594524 2026-02-17T21:17:19.361 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.361 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.361 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.361 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:17:19.361 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:17:19.361 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.361 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:17:19.362 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6804", 2026-02-17T21:17:19.362 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 507594524 2026-02-17T21:17:19.362 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.362 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.362 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:17:19.362 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6805", 2026-02-17T21:17:19.362 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 507594524 2026-02-17T21:17:19.362 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.362 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.362 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.362 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:17:19.362 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:17:19.362 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.363 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:17:19.363 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6808", 2026-02-17T21:17:19.363 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 507594524 2026-02-17T21:17:19.363 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.363 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.363 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:17:19.363 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6809", 2026-02-17T21:17:19.363 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 507594524 2026-02-17T21:17:19.363 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.363 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.363 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.363 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:17:19.363 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:17:19.364 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.364 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:17:19.364 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6806", 2026-02-17T21:17:19.364 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 507594524 2026-02-17T21:17:19.364 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.364 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.364 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:17:19.364 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6807", 2026-02-17T21:17:19.364 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 507594524 2026-02-17T21:17:19.364 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.364 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.364 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.364 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6803/507594524", 2026-02-17T21:17:19.365 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6805/507594524", 2026-02-17T21:17:19.365 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6809/507594524", 2026-02-17T21:17:19.365 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6807/507594524", 2026-02-17T21:17:19.365 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:17:19.365 INFO:tasks.workunit.client.0.trial031.stdout: "exists", 2026-02-17T21:17:19.365 INFO:tasks.workunit.client.0.trial031.stdout: "up" 2026-02-17T21:17:19.365 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.365 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.365 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.365 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:17:19.365 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "ee415559-573a-4cec-9568-aa72c415cf6a", 2026-02-17T21:17:19.365 INFO:tasks.workunit.client.0.trial031.stdout: "up": 0, 2026-02-17T21:17:19.365 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:17:19.366 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:17:19.366 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:17:19.366 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 0, 2026-02-17T21:17:19.366 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 0, 2026-02-17T21:17:19.366 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 9, 2026-02-17T21:17:19.366 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 20, 2026-02-17T21:17:19.366 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 28, 2026-02-17T21:17:19.366 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:17:19.366 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:17:19.366 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:17:19.366 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.366 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:17:19.366 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6810", 2026-02-17T21:17:19.367 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2275153387 2026-02-17T21:17:19.367 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.367 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.367 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:17:19.367 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6811", 2026-02-17T21:17:19.367 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2275153387 2026-02-17T21:17:19.367 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.367 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.367 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.367 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:17:19.367 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:17:19.367 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.367 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:17:19.368 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6812", 2026-02-17T21:17:19.368 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2275153387 2026-02-17T21:17:19.368 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.368 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.368 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:17:19.368 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6813", 2026-02-17T21:17:19.368 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2275153387 2026-02-17T21:17:19.368 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.368 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.368 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.368 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:17:19.368 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:17:19.368 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.369 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:17:19.369 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6816", 2026-02-17T21:17:19.369 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2275153387 2026-02-17T21:17:19.369 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.369 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.369 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:17:19.369 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6817", 2026-02-17T21:17:19.369 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2275153387 2026-02-17T21:17:19.369 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.369 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.369 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.369 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:17:19.369 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:17:19.370 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.370 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:17:19.370 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6814", 2026-02-17T21:17:19.370 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2275153387 2026-02-17T21:17:19.370 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.370 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.370 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:17:19.370 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6815", 2026-02-17T21:17:19.370 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2275153387 2026-02-17T21:17:19.370 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.370 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.370 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.370 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6811/2275153387", 2026-02-17T21:17:19.370 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6813/2275153387", 2026-02-17T21:17:19.371 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2275153387", 2026-02-17T21:17:19.371 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2275153387", 2026-02-17T21:17:19.371 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:17:19.371 INFO:tasks.workunit.client.0.trial031.stdout: "exists" 2026-02-17T21:17:19.371 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.371 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.371 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.371 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:17:19.371 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "f52fba25-8021-4172-b9c1-88d5728ad329", 2026-02-17T21:17:19.371 INFO:tasks.workunit.client.0.trial031.stdout: "up": 1, 2026-02-17T21:17:19.371 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:17:19.371 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:17:19.371 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:17:19.372 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 13, 2026-02-17T21:17:19.372 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 26, 2026-02-17T21:17:19.372 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 35, 2026-02-17T21:17:19.372 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 0, 2026-02-17T21:17:19.372 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 29, 2026-02-17T21:17:19.372 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:17:19.372 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:17:19.372 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:17:19.372 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.372 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:17:19.372 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6810", 2026-02-17T21:17:19.372 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 850722593 2026-02-17T21:17:19.372 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.373 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.373 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:17:19.373 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6811", 2026-02-17T21:17:19.373 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 850722593 2026-02-17T21:17:19.373 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.373 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.373 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.373 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:17:19.373 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:17:19.373 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.373 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:17:19.373 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6812", 2026-02-17T21:17:19.373 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 850722593 2026-02-17T21:17:19.374 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.374 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.374 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:17:19.374 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6813", 2026-02-17T21:17:19.374 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 850722593 2026-02-17T21:17:19.374 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.374 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.374 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.374 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:17:19.374 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:17:19.374 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.374 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:17:19.374 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6816", 2026-02-17T21:17:19.374 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 850722593 2026-02-17T21:17:19.374 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.375 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.375 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:17:19.375 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6817", 2026-02-17T21:17:19.375 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 850722593 2026-02-17T21:17:19.375 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.375 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.375 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.375 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:17:19.375 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:17:19.375 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.375 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:17:19.375 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6814", 2026-02-17T21:17:19.375 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 850722593 2026-02-17T21:17:19.375 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.375 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.376 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:17:19.376 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6815", 2026-02-17T21:17:19.376 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 850722593 2026-02-17T21:17:19.376 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.376 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.376 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.376 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6811/850722593", 2026-02-17T21:17:19.376 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6813/850722593", 2026-02-17T21:17:19.376 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6817/850722593", 2026-02-17T21:17:19.376 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6815/850722593", 2026-02-17T21:17:19.376 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:17:19.376 INFO:tasks.workunit.client.0.trial031.stdout: "exists", 2026-02-17T21:17:19.377 INFO:tasks.workunit.client.0.trial031.stdout: "up" 2026-02-17T21:17:19.377 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.377 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.377 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.377 INFO:tasks.workunit.client.0.trial031.stdout: "osd_xinfo": [ 2026-02-17T21:17:19.377 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.377 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:17:19.377 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:16:36.727841+0000", 2026-02-17T21:17:19.377 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:17:19.377 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:17:19.377 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:17:19.377 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:17:19.377 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-17T21:17:19.378 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:17:19.378 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.378 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.378 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:17:19.378 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:16:36.993021+0000", 2026-02-17T21:17:19.378 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:17:19.378 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:17:19.378 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:17:19.378 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:17:19.378 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-17T21:17:19.378 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:17:19.378 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.378 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.378 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:17:19.378 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:16:37.296328+0000", 2026-02-17T21:17:19.379 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:17:19.379 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:17:19.379 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:17:19.379 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:17:19.379 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-17T21:17:19.379 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:17:19.379 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.379 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.379 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap": [], 2026-02-17T21:17:19.379 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap_items": [], 2026-02-17T21:17:19.379 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap_primaries": [], 2026-02-17T21:17:19.379 INFO:tasks.workunit.client.0.trial031.stdout: "pg_temp": [], 2026-02-17T21:17:19.379 INFO:tasks.workunit.client.0.trial031.stdout: "primary_temp": [], 2026-02-17T21:17:19.379 INFO:tasks.workunit.client.0.trial031.stdout: "blocklist": {}, 2026-02-17T21:17:19.379 INFO:tasks.workunit.client.0.trial031.stdout: "range_blocklist": {}, 2026-02-17T21:17:19.380 INFO:tasks.workunit.client.0.trial031.stdout: "erasure_code_profiles": { 2026-02-17T21:17:19.380 INFO:tasks.workunit.client.0.trial031.stdout: "default": { 2026-02-17T21:17:19.380 INFO:tasks.workunit.client.0.trial031.stdout: "k": "2", 2026-02-17T21:17:19.380 INFO:tasks.workunit.client.0.trial031.stdout: "m": "2", 2026-02-17T21:17:19.380 INFO:tasks.workunit.client.0.trial031.stdout: "plugin": "jerasure", 2026-02-17T21:17:19.380 INFO:tasks.workunit.client.0.trial031.stdout: "technique": "reed_sol_van" 2026-02-17T21:17:19.380 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.380 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.380 INFO:tasks.workunit.client.0.trial031.stdout: "removed_snaps_queue": [], 2026-02-17T21:17:19.380 INFO:tasks.workunit.client.0.trial031.stdout: "new_removed_snaps": [], 2026-02-17T21:17:19.380 INFO:tasks.workunit.client.0.trial031.stdout: "new_purged_snaps": [], 2026-02-17T21:17:19.380 INFO:tasks.workunit.client.0.trial031.stdout: "crush_node_flags": {}, 2026-02-17T21:17:19.380 INFO:tasks.workunit.client.0.trial031.stdout: "device_class_flags": {}, 2026-02-17T21:17:19.380 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode": { 2026-02-17T21:17:19.381 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode_enabled": false, 2026-02-17T21:17:19.381 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_bucket_count": 0, 2026-02-17T21:17:19.381 INFO:tasks.workunit.client.0.trial031.stdout: "degraded_stretch_mode": 0, 2026-02-17T21:17:19.381 INFO:tasks.workunit.client.0.trial031.stdout: "recovering_stretch_mode": 0, 2026-02-17T21:17:19.381 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode_bucket": 0 2026-02-17T21:17:19.381 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.381 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.381 INFO:tasks.workunit.client.0.trial031.stdout: "osd_metadata": [ 2026-02-17T21:17:19.381 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.381 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:17:19.381 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:17:19.381 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6804/507594524,v1:127.0.0.1:6805/507594524]", 2026-02-17T21:17:19.381 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:17:19.381 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:17:19.381 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:17:19.382 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:15:47.584423Z", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6802/507594524,v1:127.0.0.1:6803/507594524]", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/507594524,v1:127.0.0.1:6809/507594524]", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/507594524,v1:127.0.0.1:6807/507594524]", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:17:19.383 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/0", 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:17:19.384 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6812/2275153387,v1:127.0.0.1:6813/2275153387]", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:17:19.385 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:15:56.824952Z", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:17:19.386 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6810/2275153387,v1:127.0.0.1:6811/2275153387]", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2275153387,v1:127.0.0.1:6817/2275153387]", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2275153387,v1:127.0.0.1:6815/2275153387]", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/1", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:17:19.387 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6812/850722593,v1:127.0.0.1:6813/850722593]", 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-17T21:17:19.388 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:16:04.852349Z", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6810/850722593,v1:127.0.0.1:6811/850722593]", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:17:19.389 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/850722593,v1:127.0.0.1:6817/850722593]", 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/850722593,v1:127.0.0.1:6815/850722593]", 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/2", 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_clean_epochs": { 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "min_last_epoch_clean": 0, 2026-02-17T21:17:19.390 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_clean": { 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: "per_pool": [] 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: "osd_epochs": [] 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_first_committed": 1, 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_last_committed": 40, 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: "crushmap": { 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: "devices": [ 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.0", 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.1", 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:17:19.391 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.2", 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: "types": [ 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 0, 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd" 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: "name": "host" 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 2, 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: "name": "chassis" 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 3, 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: "name": "rack" 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.392 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 4, 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: "name": "row" 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 5, 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: "name": "pdu" 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 6, 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: "name": "pod" 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 7, 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: "name": "room" 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 8, 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: "name": "datacenter" 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 9, 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: "name": "zone" 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.393 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 10, 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "name": "region" 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "name": "root" 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "buckets": [ 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "id": -1, 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "name": "default", 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "root", 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "id": -3, 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:17:19.394 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: "id": -2, 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: "name": "default~ssd", 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "root", 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: "id": -4, 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: "id": -3, 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: "name": "trial031", 2026-02-17T21:17:19.395 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "host", 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 1 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 2 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "id": -4, 2026-02-17T21:17:19.396 INFO:tasks.workunit.client.0.trial031.stdout: "name": "trial031~ssd", 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "host", 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 1 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 2 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.397 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: "rules": [ 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: "rule_id": 0, 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: "rule_name": "replicated_rule", 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: "type": 1, 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: "steps": [ 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: "op": "take", 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: "item": -1, 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: "item_name": "default" 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: "op": "choose_firstn", 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: "num": 0, 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: "type": "osd" 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: "op": "emit" 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: "tunables": { 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: "choose_local_tries": 0, 2026-02-17T21:17:19.398 INFO:tasks.workunit.client.0.trial031.stdout: "choose_local_fallback_tries": 0, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "choose_total_tries": 50, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_descend_once": 1, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_vary_r": 1, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_stable": 1, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "msr_descents": 100, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "msr_collision_tries": 100, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "straw_calc_version": 1, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "allowed_bucket_algs": 54, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "profile": "jewel", 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "optimal_tunables": 1, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "legacy_tunables": 0, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "minimum_required_version": "jewel", 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables": 1, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables2": 1, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "has_v2_rules": 0, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables3": 1, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "has_v3_rules": 0, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "has_v4_buckets": 1, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables5": 1, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "has_v5_rules": 0, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "has_msr_rules": 0 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "choose_args": {} 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.399 INFO:tasks.workunit.client.0.trial031.stdout: "fsmap": { 2026-02-17T21:17:19.400 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 1, 2026-02-17T21:17:19.400 INFO:tasks.workunit.client.0.trial031.stdout: "btime": "2026-02-17T21:15:44:973143+0000", 2026-02-17T21:17:19.400 INFO:tasks.workunit.client.0.trial031.stderr:report 1064837833 2026-02-17T21:17:19.400 INFO:tasks.workunit.client.0.trial031.stdout: "default_fscid": -1, 2026-02-17T21:17:19.400 INFO:tasks.workunit.client.0.trial031.stdout: "compat": { 2026-02-17T21:17:19.400 INFO:tasks.workunit.client.0.trial031.stdout: "compat": {}, 2026-02-17T21:17:19.400 INFO:tasks.workunit.client.0.trial031.stdout: "ro_compat": {}, 2026-02-17T21:17:19.400 INFO:tasks.workunit.client.0.trial031.stdout: "incompat": { 2026-02-17T21:17:19.400 INFO:tasks.workunit.client.0.trial031.stdout: "feature_1": "base v0.20", 2026-02-17T21:17:19.400 INFO:tasks.workunit.client.0.trial031.stdout: "feature_2": "client writeable ranges", 2026-02-17T21:17:19.400 INFO:tasks.workunit.client.0.trial031.stdout: "feature_3": "default file layouts on dirs", 2026-02-17T21:17:19.400 INFO:tasks.workunit.client.0.trial031.stdout: "feature_4": "dir inode in separate object", 2026-02-17T21:17:19.400 INFO:tasks.workunit.client.0.trial031.stdout: "feature_5": "mds uses versioned encoding", 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "feature_8": "no anchor table", 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "feature_9": "file layout v2", 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "feature_10": "snaprealm v2", 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "feature_11": "minor log segments", 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "feature_12": "quiesce subvolumes" 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "feature_flags": { 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "enable_multiple": true, 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "ever_enabled_multiple": true 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "standbys": [], 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "filesystems": [] 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "mdsmap_first_committed": 1, 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "mdsmap_last_committed": 1, 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "auth": { 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "first_committed": 1, 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "last_committed": 5, 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "num_secrets": 3 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg": 1, 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_active": 1, 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_unknown": 0, 2026-02-17T21:17:19.401 INFO:tasks.workunit.client.0.trial031.stdout: "num_osd": 3, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "pool_sum": { 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "stat_sum": { 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes": 70600, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects": 100, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_clones": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_copies": 300, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_degraded": 100, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_misplaced": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_unfound": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_dirty": 100, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_whiteouts": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_read": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_read_kb": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_write": 100, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_write_kb": 100, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_scrub_errors": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_deep_scrub_errors": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_recovered": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_recovered": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_keys_recovered": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_omap": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush": 0, 2026-02-17T21:17:19.402 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_kb": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_kb": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "num_promote": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_high": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_low": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_some": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_full": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_pinned": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "num_legacy_snapsets": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "num_large_omap_objects": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_manifest": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_bytes": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_keys": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_repaired": 0 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "store_stats": { 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "total": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "available": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 0, 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 0 2026-02-17T21:17:19.403 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "log_size": 16, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "ondisk_log_size": 16, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "up": 2, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "acting": 2, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "num_store_stats": 0 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "osd_sum": { 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 0, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 0, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "num_pgs": 3, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "num_osds": 3, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_osds": 3, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "kb": 314572800, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used": 83896, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_data": 2072, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_omap": 10, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_meta": 81717, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "kb_avail": 314488904, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "statfs": { 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "total": 322122547200, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "available": 322036637696, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 2121728, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 479792, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:17:19.404 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 10383, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 83679089 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "hb_peers": [], 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "snap_trim_queue_len": 0, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "num_snap_trimming": 0, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "num_shards_repaired": 1, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "op_queue_age_hist": { 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "histogram": [], 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "upper_bound": 1 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "perf_stat": { 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ms": 0, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ms": 0, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ns": 0, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ns": 0 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "alerts": [], 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "network_ping_times": [] 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "osd_sum_by_class": { 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "ssd": { 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 0, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 0, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "num_pgs": 3, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "num_osds": 3, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_osds": 3, 2026-02-17T21:17:19.405 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "kb": 314572800, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used": 83896, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_data": 2072, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_omap": 10, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_meta": 81717, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "kb_avail": 314488904, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "statfs": { 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "total": 322122547200, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "available": 322036637696, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 2121728, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 479792, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 10383, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 83679089 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "hb_peers": [], 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "snap_trim_queue_len": 0, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "num_snap_trimming": 0, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "num_shards_repaired": 1, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "op_queue_age_hist": { 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "histogram": [], 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "upper_bound": 1 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "perf_stat": { 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ms": 0, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ms": 0, 2026-02-17T21:17:19.406 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ns": 0, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ns": 0 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "alerts": [], 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "network_ping_times": [] 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "pool_stats": [ 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "poolid": 1, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg": 1, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "stat_sum": { 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes": 70600, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects": 100, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_clones": 0, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_copies": 300, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing": 0, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_degraded": 100, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_misplaced": 0, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_unfound": 0, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_dirty": 100, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_whiteouts": 0, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_read": 0, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_read_kb": 0, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_write": 100, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_write_kb": 100, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_scrub_errors": 0, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_deep_scrub_errors": 0, 2026-02-17T21:17:19.407 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_recovered": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_recovered": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_keys_recovered": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_omap": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_kb": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_kb": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_promote": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_high": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_low": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_some": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_full": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_pinned": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_legacy_snapsets": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_large_omap_objects": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_manifest": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_bytes": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_keys": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_repaired": 0 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "store_stats": { 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "total": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "available": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 1228800, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 211800, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 0, 2026-02-17T21:17:19.408 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 0 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "log_size": 16, 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "ondisk_log_size": 16, 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "up": 2, 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "acting": 2, 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "num_store_stats": 3 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "osd_stats": [ 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 141733920777 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 38654705677 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 150323855368 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_by_state": [ 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "state": "active+undersized+degraded", 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "num": 1 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_by_osd": [ 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.409 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 1, 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 1, 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 1, 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 0, 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snaps": [ 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "pool": 1, 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snaps": {} 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "servicemap": { 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 4, 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:17:03.454075+0000", 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "services": { 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "osd": { 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "daemons": { 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "summary": "", 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "0": { 2026-02-17T21:17:19.410 INFO:tasks.workunit.client.0.trial031.stdout: "start_epoch": 0, 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "start_stamp": "0.000000", 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "gid": 0, 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "metadata": {}, 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "task_status": {} 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "2": { 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "start_epoch": 0, 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "start_stamp": "0.000000", 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "gid": 0, 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "metadata": {}, 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "task_status": {} 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "mgrstat_first_committed": 1, 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "mgrstat_last_committed": 44, 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "logm_first_committed": 1, 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "logm_last_committed": 235, 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "paxos": { 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "first_committed": 1, 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "last_committed": 330, 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "last_pn": 0, 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: "accepted_pn": 0 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:17:19.411 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:17:19.412 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-17T21:17:19.412 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:153: TEST_divergent: expr 2 + 2 2026-02-17T21:17:19.412 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:153: TEST_divergent: objname=existing_4 2026-02-17T21:17:19.412 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:154: TEST_divergent: echo 'writing non-divergent object existing_4' 2026-02-17T21:17:19.412 INFO:tasks.workunit.client.0.trial031.stdout:writing non-divergent object existing_4 2026-02-17T21:17:19.412 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:155: TEST_divergent: ceph pg dump pgs 2026-02-17T21:17:19.549 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:17:19.549 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70600 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:17:05.047615+0000 26'100 40:139 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:16:13.211711+0000 0'0 2026-02-17T21:16:13.211711+0000 0 0 periodic scrub scheduled @ 2026-02-18T21:57:32.924353+0000 0 0 2026-02-17T21:17:19.549 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:17:19.549 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:17:19.549 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:17:19.557 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:156: TEST_divergent: rados -p test put existing_4 /etc/resolv.conf 2026-02-17T21:17:19.588 INFO:tasks.workunit.client.0.trial031.stdout:delay recovery 2026-02-17T21:17:19.588 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:159: TEST_divergent: echo 'delay recovery' 2026-02-17T21:17:19.588 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:160: TEST_divergent: ceph pg dump pgs 2026-02-17T21:17:19.778 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:17:19.778 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70600 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:17:05.047615+0000 26'100 40:139 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:16:13.211711+0000 0'0 2026-02-17T21:16:13.211711+0000 0 0 periodic scrub scheduled @ 2026-02-18T21:57:32.924353+0000 0 0 2026-02-17T21:17:19.778 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:17:19.778 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:17:19.779 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:17:19.786 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: for i in $non_divergent 2026-02-17T21:17:19.787 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: get_asok_path osd.0 2026-02-17T21:17:19.787 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T21:17:19.788 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T21:17:19.788 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:17:19.788 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:17:19.788 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:17:19.788 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.0.asok 2026-02-17T21:17:19.788 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: CEPH_ARGS= 2026-02-17T21:17:19.789 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.0.asok set_recovery_delay 100000 2026-02-17T21:17:19.844 INFO:tasks.workunit.client.0.trial031.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-17T21:17:19.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: for i in $non_divergent 2026-02-17T21:17:19.851 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: get_asok_path osd.2 2026-02-17T21:17:19.851 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-17T21:17:19.851 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-17T21:17:19.851 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:17:19.851 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:17:19.851 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:17:19.851 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.2.asok 2026-02-17T21:17:19.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: CEPH_ARGS= 2026-02-17T21:17:19.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.2.asok set_recovery_delay 100000 2026-02-17T21:17:19.904 INFO:tasks.workunit.client.0.trial031.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-17T21:17:19.910 INFO:tasks.workunit.client.0.trial031.stdout:revive divergent 1 2026-02-17T21:17:19.910 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:167: TEST_divergent: echo 'revive divergent 1' 2026-02-17T21:17:19.910 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:168: TEST_divergent: ceph pg dump pgs 2026-02-17T21:17:20.115 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:17:20.115 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70600 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:17:05.047615+0000 26'100 40:139 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:16:13.211711+0000 0'0 2026-02-17T21:16:13.211711+0000 0 0 periodic scrub scheduled @ 2026-02-18T21:57:32.924353+0000 0 0 2026-02-17T21:17:20.115 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:17:20.115 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:17:20.116 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:17:20.123 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:169: TEST_divergent: ceph osd set noup 2026-02-17T21:17:20.445 INFO:tasks.workunit.client.0.trial031.stderr:noup is set 2026-02-17T21:17:20.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:170: TEST_divergent: activate_osd td/divergent-priors 1 2026-02-17T21:17:20.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:17:20.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:17:20.459 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T21:17:20.459 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:17:20.459 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-17T21:17:20.459 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:17:20.459 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:17:20.459 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:17:20.459 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:17:20.459 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-17T21:17:20.459 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-17T21:17:20.459 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:17:20.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:17:20.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:17:20.460 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:17:20.460 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:17:20.460 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:17:20.460 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:17:20.460 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:17:20.460 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:17:20.460 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:17:20.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:17:20.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:17:20.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:17:20.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:17:20.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:17:20.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:17:20.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:17:20.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:17:20.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:17:20.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:17:20.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-17T21:17:20.462 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:17:20.463 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T21:17:20.463 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:17:20.463 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-17T21:17:20.463 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T21:17:20.463 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:17:20.463 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:17:20.463 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:17:20.477 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:17:20.471+0000 7f263b3b18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:17:20.478 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:17:20.471+0000 7f263b3b18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:17:20.479 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:17:20.471+0000 7f263b3b18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:17:20.658 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:880: activate_osd: return 0 2026-02-17T21:17:20.658 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:171: TEST_divergent: sleep 5 2026-02-17T21:17:21.705 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:17:21.699+0000 7f263b3b18c0 -1 Falling back to public interface 2026-02-17T21:17:24.678 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:17:24.671+0000 7f263b3b18c0 -1 osd.1 26 log_to_monitors true 2026-02-17T21:17:25.659 INFO:tasks.workunit.client.0.trial031.stdout:delay recovery divergent 2026-02-17T21:17:25.659 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:173: TEST_divergent: echo 'delay recovery divergent' 2026-02-17T21:17:25.659 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:174: TEST_divergent: ceph pg dump pgs 2026-02-17T21:17:25.853 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:17:25.853 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70861 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:17:05.047615+0000 40'101 42:143 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:16:13.211711+0000 0'0 2026-02-17T21:16:13.211711+0000 0 0 periodic scrub scheduled @ 2026-02-18T21:57:32.924353+0000 0 0 2026-02-17T21:17:25.853 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:17:25.854 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:17:25.854 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:17:25.862 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:175: TEST_divergent: get_asok_path osd.1 2026-02-17T21:17:25.862 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T21:17:25.862 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T21:17:25.862 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:17:25.862 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:17:25.862 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:17:25.863 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.1.asok 2026-02-17T21:17:25.863 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:175: TEST_divergent: CEPH_ARGS= 2026-02-17T21:17:25.863 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:175: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.1.asok set_recovery_delay 100000 2026-02-17T21:17:25.918 INFO:tasks.workunit.client.0.trial031.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-17T21:17:25.923 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:177: TEST_divergent: ceph osd unset noup 2026-02-17T21:17:26.242 INFO:tasks.workunit.client.0.trial031.stderr:noup is unset 2026-02-17T21:17:26.256 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:179: TEST_divergent: wait_for_osd up 0 2026-02-17T21:17:26.256 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:17:26.256 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:17:26.256 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:17:26.256 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:17:26.256 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:17:26.256 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:17:26.257 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:17:26.257 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:17:26.257 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:17:26.460 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 33 up_thru 39 down_at 27 last_clean_interval [5,26) [v2:127.0.0.1:6802/507594524,v1:127.0.0.1:6803/507594524] [v2:127.0.0.1:6804/507594524,v1:127.0.0.1:6805/507594524] exists,up f21d462f-1bd6-4e5f-afc1-346559015142 2026-02-17T21:17:26.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:17:26.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:17:26.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:17:26.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:180: TEST_divergent: wait_for_osd up 1 2026-02-17T21:17:26.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:17:26.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:17:26.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:17:26.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:17:26.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:17:26.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:17:26.463 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:17:26.463 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:17:26.463 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:17:26.661 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 45 up_thru 45 down_at 28 last_clean_interval [9,26) [v2:127.0.0.1:6818/812989745,v1:127.0.0.1:6819/812989745] [v2:127.0.0.1:6820/812989745,v1:127.0.0.1:6821/812989745] exists,up ee415559-573a-4cec-9568-aa72c415cf6a 2026-02-17T21:17:26.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:17:26.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:17:26.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:17:26.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:181: TEST_divergent: wait_for_osd up 2 2026-02-17T21:17:26.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:17:26.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:17:26.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:17:26.663 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:17:26.663 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:17:26.663 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:17:26.663 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:17:26.663 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:17:26.664 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:17:26.864 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 35 up_thru 0 down_at 29 last_clean_interval [13,26) [v2:127.0.0.1:6810/850722593,v1:127.0.0.1:6811/850722593] [v2:127.0.0.1:6812/850722593,v1:127.0.0.1:6813/850722593] exists,up f52fba25-8021-4172-b9c1-88d5728ad329 2026-02-17T21:17:26.865 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:17:26.865 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:17:26.865 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:17:26.865 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:183: TEST_divergent: ceph pg dump pgs 2026-02-17T21:17:27.057 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:17:27.057 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70861 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:17:05.047615+0000 40'101 42:143 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:16:13.211711+0000 0'0 2026-02-17T21:16:13.211711+0000 0 0 periodic scrub scheduled @ 2026-02-18T21:57:32.924353+0000 0 0 2026-02-17T21:17:27.058 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:17:27.058 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:17:27.058 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:17:27.065 INFO:tasks.workunit.client.0.trial031.stdout:wait for peering 2026-02-17T21:17:27.066 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:184: TEST_divergent: echo 'wait for peering' 2026-02-17T21:17:27.066 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:185: TEST_divergent: ceph pg dump pgs 2026-02-17T21:17:27.265 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:17:27.266 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70861 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:17:05.047615+0000 40'101 42:143 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:16:13.211711+0000 0'0 2026-02-17T21:16:13.211711+0000 0 0 periodic scrub scheduled @ 2026-02-18T21:57:32.924353+0000 0 0 2026-02-17T21:17:27.266 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:17:27.266 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:17:27.266 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:17:27.276 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:186: TEST_divergent: rados -p test put foo /etc/fstab 2026-02-17T21:17:27.299 INFO:tasks.workunit.client.0.trial031.stdout:killing divergent 1 2026-02-17T21:17:27.300 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:188: TEST_divergent: echo 'killing divergent 1' 2026-02-17T21:17:27.300 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:189: TEST_divergent: ceph pg dump pgs 2026-02-17T21:17:27.505 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:17:27.505 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70861 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:17:05.047615+0000 40'101 42:143 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:16:13.211711+0000 0'0 2026-02-17T21:16:13.211711+0000 0 0 periodic scrub scheduled @ 2026-02-18T21:57:32.924353+0000 0 0 2026-02-17T21:17:27.505 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:17:27.505 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:17:27.505 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:17:27.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:190: TEST_divergent: kill_daemons td/divergent-priors KILL osd.1 2026-02-17T21:17:27.514 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:17:27.514 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:17:27.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:17:27.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:17:27.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:17:27.621 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:17:27.622 INFO:tasks.workunit.client.0.trial031.stdout:reviving divergent 1 2026-02-17T21:17:27.622 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:192: TEST_divergent: echo 'reviving divergent 1' 2026-02-17T21:17:27.622 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:193: TEST_divergent: ceph pg dump pgs 2026-02-17T21:17:27.814 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:17:27.814 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70861 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:17:05.047615+0000 40'101 42:143 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:16:13.211711+0000 0'0 2026-02-17T21:16:13.211711+0000 0 0 periodic scrub scheduled @ 2026-02-18T21:57:32.924353+0000 0 0 2026-02-17T21:17:27.814 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:17:27.814 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:17:27.815 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:17:27.826 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:194: TEST_divergent: activate_osd td/divergent-priors 1 2026-02-17T21:17:27.826 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:17:27.826 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:17:27.826 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T21:17:27.826 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:17:27.827 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-17T21:17:27.827 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:17:27.827 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:17:27.827 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:17:27.827 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:17:27.827 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-17T21:17:27.827 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-17T21:17:27.827 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:17:27.827 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:17:27.827 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:17:27.828 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:17:27.828 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:17:27.828 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:17:27.828 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:17:27.828 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:17:27.828 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:17:27.828 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:17:27.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:17:27.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:17:27.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:17:27.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:17:27.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:17:27.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:17:27.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:17:27.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:17:27.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:17:27.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:17:27.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-17T21:17:27.829 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:17:27.830 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T21:17:27.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:17:27.831 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-17T21:17:27.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T21:17:27.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:17:27.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:17:27.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:17:27.852 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:17:27.847+0000 7f9acd38c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:17:27.854 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:17:27.847+0000 7f9acd38c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:17:27.855 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:17:27.847+0000 7f9acd38c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:17:28.036 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T21:17:28.036 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:17:28.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:17:28.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:17:28.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:17:28.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:17:28.037 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:17:28.038 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:17:28.038 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:17:28.038 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:17:28.245 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 45 up_thru 45 down_at 28 last_clean_interval [9,26) [v2:127.0.0.1:6818/812989745,v1:127.0.0.1:6819/812989745] [v2:127.0.0.1:6820/812989745,v1:127.0.0.1:6821/812989745] exists,up ee415559-573a-4cec-9568-aa72c415cf6a 2026-02-17T21:17:28.246 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:17:28.246 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:17:28.246 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:17:28.246 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:196: TEST_divergent: sleep 20 2026-02-17T21:17:29.330 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:17:29.323+0000 7f9acd38c8c0 -1 Falling back to public interface 2026-02-17T21:17:32.320 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:17:32.315+0000 7f9acd38c8c0 -1 osd.1 46 log_to_monitors true 2026-02-17T21:17:48.246 INFO:tasks.workunit.client.0.trial031.stdout:allowing recovery 2026-02-17T21:17:48.246 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:198: TEST_divergent: echo 'allowing recovery' 2026-02-17T21:17:48.246 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:199: TEST_divergent: ceph pg dump pgs 2026-02-17T21:17:48.434 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:17:48.434 INFO:tasks.workunit.client.0.trial031.stdout:1.0 101 0 0 0 0 71567 0 0 18 84 18 active+clean 2026-02-17T21:17:32.854858+0000 46'102 52:162 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:16:13.211711+0000 0'0 2026-02-17T21:16:13.211711+0000 0 0 periodic scrub scheduled @ 2026-02-18T21:56:09.031929+0000 0 0 2026-02-17T21:17:48.434 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:17:48.434 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:17:48.434 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:17:48.442 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: for i in $osds 2026-02-17T21:17:48.442 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:203: TEST_divergent: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-17T21:17:48.512 INFO:tasks.workunit.client.0.trial031.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-17T21:17:48.519 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: for i in $osds 2026-02-17T21:17:48.519 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:203: TEST_divergent: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-17T21:17:48.601 INFO:tasks.workunit.client.0.trial031.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-17T21:17:48.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: for i in $osds 2026-02-17T21:17:48.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:203: TEST_divergent: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-17T21:17:48.688 INFO:tasks.workunit.client.0.trial031.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-17T21:17:48.695 INFO:tasks.workunit.client.0.trial031.stdout:reading divergent objects 2026-02-17T21:17:48.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: echo 'reading divergent objects' 2026-02-17T21:17:48.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:207: TEST_divergent: ceph pg dump pgs 2026-02-17T21:17:48.886 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:17:48.886 INFO:tasks.workunit.client.0.trial031.stdout:1.0 101 0 0 0 0 71567 0 0 18 84 18 active+clean 2026-02-17T21:17:32.854858+0000 46'102 52:162 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:16:13.211711+0000 0'0 2026-02-17T21:16:13.211711+0000 0 0 periodic scrub scheduled @ 2026-02-18T21:56:09.031929+0000 0 0 2026-02-17T21:17:48.886 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:17:48.886 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:17:48.886 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:17:48.894 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: expr 2 + 2 2026-02-17T21:17:48.896 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: seq 1 4 2026-02-17T21:17:48.897 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:17:48.897 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_1 td/divergent-priors/existing 2026-02-17T21:17:48.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:17:48.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_2 td/divergent-priors/existing 2026-02-17T21:17:48.954 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:17:48.954 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_3 td/divergent-priors/existing 2026-02-17T21:17:48.982 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:17:48.982 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_4 td/divergent-priors/existing 2026-02-17T21:17:49.011 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:212: TEST_divergent: rm -f td/divergent-priors/existing 2026-02-17T21:17:49.012 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:214: TEST_divergent: find td/divergent-priors -name '*osd*log' 2026-02-17T21:17:49.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:214: TEST_divergent: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-17T21:17:49.019 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:17:26.343+0000 7f261932f640 20 _merge_object_divergent_entries: merging hoid 1:c156033e:::existing_100:head entries: 26'101 (26'100) modify 1:c156033e:::existing_100:head by client.4548.0:1 2026-02-17T21:16:25.849181+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: true, new_object: false 2026-02-17T21:17:49.019 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:17:26.343+0000 7f261932f640 20 _merge_object_divergent_entries: keeping 26'101 (26'100) modify 1:c156033e:::existing_100:head by client.4548.0:1 2026-02-17T21:16:25.849181+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: true, new_object: false 2026-02-17T21:17:49.019 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:17:26.343+0000 7f261932f640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-17T21:17:49.019 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:17:26.343+0000 7f261932f640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head prior_version: 26'100 first_divergent_update: 26'101 last_divergent_update: 26'101 2026-02-17T21:17:49.019 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:17:26.343+0000 7f261932f640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head has no more recent entries in log 2026-02-17T21:17:49.019 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:17:26.343+0000 7f261932f640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head must be rolled back or recovered, attempting to rollback 2026-02-17T21:17:49.019 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:17:26.343+0000 7f261932f640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head olog_can_rollback_to: 26'101 2026-02-17T21:17:49.019 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:17:26.343+0000 7f261932f640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot rollback 26'101 (26'100) modify 1:c156033e:::existing_100:head by client.4548.0:1 2026-02-17T21:16:25.849181+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: true, new_object: false 2026-02-17T21:17:49.020 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:17:26.343+0000 7f261932f640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot roll back, removing and adding to missing 2026-02-17T21:17:49.020 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:216: TEST_divergent: find td/divergent-priors -name '*osd*log' 2026-02-17T21:17:49.020 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:216: TEST_divergent: grep -q '_merge_object_divergent_entries.*cannot roll back, removing and adding to missing' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-17T21:17:49.023 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:221: TEST_divergent: echo success 2026-02-17T21:17:49.024 INFO:tasks.workunit.client.0.trial031.stdout:success 2026-02-17T21:17:49.024 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:223: TEST_divergent: delete_pool test 2026-02-17T21:17:49.024 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=test 2026-02-17T21:17:49.024 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-17T21:17:49.272 INFO:tasks.workunit.client.0.trial031.stderr:pool 'test' does not exist 2026-02-17T21:17:49.280 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:224: TEST_divergent: kill_daemons td/divergent-priors 2026-02-17T21:17:49.281 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:17:49.281 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:17:49.281 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:17:49.282 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:17:49.282 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:17:57.598 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:17:57.598 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-17T21:17:57.598 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-17T21:17:57.598 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:17:57.598 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-17T21:17:57.599 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:17:57.599 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:17:57.599 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:17:57.599 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:17:57.599 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:17:57.603 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:17:57.604 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:17:57.604 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:17:57.605 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:17:57.606 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:17:57.606 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:17:57.606 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:17:57.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:17:57.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:17:57.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:17:57.608 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:17:57.609 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:17:57.610 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:17:57.610 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-17T21:17:57.627 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:17:57.627 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:17:57.627 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:17:57.627 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.16076 2026-02-17T21:17:57.630 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:17:57.630 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:17:57.630 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-17T21:17:57.630 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-17T21:17:57.630 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2026-02-17T21:17:57.630 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2026-02-17T21:17:57.630 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-17T21:17:57.630 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:17:57.630 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-17T21:17:57.630 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:17:57.631 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:17:57.631 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:17:57.631 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:17:57.631 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:17:57.632 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:17:57.632 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:17:57.633 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:17:57.633 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:17:57.634 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:17:57.634 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:17:57.634 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:17:57.635 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:17:57.636 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:17:57.636 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:17:57.636 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:17:57.637 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:17:57.639 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:17:57.639 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-17T21:17:57.640 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:17:57.640 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:17:57.640 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:17:57.640 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.16076 2026-02-17T21:17:57.641 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:17:57.641 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:17:57.641 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2026-02-17T21:17:57.643 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T21:17:57.643 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:17:57.643 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:17:57.643 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.16076 2026-02-17T21:17:57.645 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T21:17:57.646 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T21:17:57.646 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T21:17:57.646 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-17T21:17:57.646 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_2 td/divergent-priors 2026-02-17T21:17:57.646 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:421: TEST_divergent_2: local dir=td/divergent-priors 2026-02-17T21:17:57.646 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:424: TEST_divergent_2: local dummyfile=/etc/fstab 2026-02-17T21:17:57.646 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:425: TEST_divergent_2: local dummyfile2=/etc/resolv.conf 2026-02-17T21:17:57.646 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:427: TEST_divergent_2: local num_osds=3 2026-02-17T21:17:57.647 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:428: TEST_divergent_2: expr 3 - 1 2026-02-17T21:17:57.647 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:428: TEST_divergent_2: seq 0 2 2026-02-17T21:17:57.648 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:428: TEST_divergent_2: local 'osds=0 2026-02-17T21:17:57.648 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:17:57.649 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:17:57.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:429: TEST_divergent_2: run_mon td/divergent-priors a 2026-02-17T21:17:57.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2026-02-17T21:17:57.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T21:17:57.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T21:17:57.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T21:17:57.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2026-02-17T21:17:57.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-17T21:17:57.679 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T21:17:57.679 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:17:57.679 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:17:57.680 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:17:57.680 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:17:57.680 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:17:57.680 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:17:57.680 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T21:17:57.712 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T21:17:57.713 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T21:17:57.713 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:17:57.713 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:17:57.713 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T21:17:57.713 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T21:17:57.715 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:17:57.715 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:17:57.715 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:17:57.715 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:17:57.715 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:17:57.716 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:17:57.716 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-mon.a.asok 2026-02-17T21:17:57.716 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:17:57.716 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.16076/ceph-mon.a.asok config get fsid 2026-02-17T21:17:57.768 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T21:17:57.769 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:17:57.769 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:17:57.769 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T21:17:57.770 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T21:17:57.770 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:17:57.770 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:17:57.770 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:17:57.770 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:17:57.770 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:17:57.771 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:17:57.771 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-mon.a.asok 2026-02-17T21:17:57.771 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:17:57.771 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.16076/ceph-mon.a.asok config get mon_host 2026-02-17T21:17:57.823 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:430: TEST_divergent_2: run_mgr td/divergent-priors x 2026-02-17T21:17:57.823 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/divergent-priors 2026-02-17T21:17:57.823 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T21:17:57.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T21:17:57.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T21:17:57.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/divergent-priors/x 2026-02-17T21:17:57.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T21:17:57.935 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T21:17:57.935 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:17:57.935 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:17:57.935 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:17:57.936 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:17:57.936 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:17:57.936 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:17:57.936 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:17:57.937 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:17:57.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:431: TEST_divergent_2: for i in $osds 2026-02-17T21:17:57.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:433: TEST_divergent_2: run_osd td/divergent-priors 0 2026-02-17T21:17:57.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-17T21:17:57.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:17:57.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T21:17:57.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:17:57.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/0 2026-02-17T21:17:57.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:17:57.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:17:57.960 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:17:57.960 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:17:57.960 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-17T21:17:57.960 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-17T21:17:57.960 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:17:57.960 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:17:57.960 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:17:57.960 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:17:57.960 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:17:57.960 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:17:57.960 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:17:57.960 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:17:57.961 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:17:57.961 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:17:57.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:17:57.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:17:57.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:17:57.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:17:57.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:17:57.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:17:57.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:17:57.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:17:57.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:17:57.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:17:57.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:17:57.962 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:17:57.962 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/0 2026-02-17T21:17:57.962 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:17:57.962 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=505cc7f4-fb42-4465-8933-5f18295caf89 2026-02-17T21:17:57.963 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 505cc7f4-fb42-4465-8933-5f18295caf89' 2026-02-17T21:17:57.963 INFO:tasks.workunit.client.0.trial031.stdout:add osd0 505cc7f4-fb42-4465-8933-5f18295caf89 2026-02-17T21:17:57.963 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:17:57.974 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAF25RpqN3lORAAe1/qdNDFcA51tKYJp8P3oQ== 2026-02-17T21:17:57.974 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAF25RpqN3lORAAe1/qdNDFcA51tKYJp8P3oQ=="}' 2026-02-17T21:17:57.974 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 505cc7f4-fb42-4465-8933-5f18295caf89 -i td/divergent-priors/0/new.json 2026-02-17T21:17:58.091 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:17:58.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/0/new.json 2026-02-17T21:17:58.100 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAF25RpqN3lORAAe1/qdNDFcA51tKYJp8P3oQ== --osd-uuid 505cc7f4-fb42-4465-8933-5f18295caf89 2026-02-17T21:17:58.118 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:17:58.112+0000 7f07ae8f48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:17:58.120 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:17:58.112+0000 7f07ae8f48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:17:58.121 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:17:58.112+0000 7f07ae8f48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:17:58.121 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:17:58.112+0000 7f07ae8f48c0 -1 bdev(0x55dd860dbc00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-17T21:17:58.121 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:17:58.112+0000 7f07ae8f48c0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-17T21:18:00.978 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-17T21:18:00.978 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:18:00.979 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T21:18:00.980 INFO:tasks.workunit.client.0.trial031.stdout:adding osd0 key to auth repository 2026-02-17T21:18:00.980 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:18:01.254 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:18:01.255 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T21:18:01.255 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:18:01.255 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:18:01.255 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:18:01.255 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:18:01.281 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:01.276+0000 7f3fd62648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:01.283 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:01.276+0000 7f3fd62648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:01.284 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:01.276+0000 7f3fd62648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:01.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T21:18:01.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:18:01.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:18:01.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:18:01.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:18:01.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:01.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:18:01.458 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:18:01.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:01.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:18:01.658 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:02.659 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:02.660 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:02.660 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:18:02.661 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:18:02.661 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:02.661 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:18:02.860 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:03.011 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:03.004+0000 7f3fd62648c0 -1 Falling back to public interface 2026-02-17T21:18:03.861 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:03.862 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:03.862 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:18:03.863 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:18:03.863 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:03.863 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:18:04.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:05.064 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:05.065 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:05.065 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:18:05.065 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:18:05.066 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:05.066 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:18:05.263 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:05.994 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:05.988+0000 7f3fd62648c0 -1 osd.0 0 log_to_monitors true 2026-02-17T21:18:06.264 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:06.264 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:06.265 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:18:06.265 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:18:06.266 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:06.266 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:18:06.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:07.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:07.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:07.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:18:07.462 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:18:07.463 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:07.463 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:18:07.664 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3695890167,v1:127.0.0.1:6803/3695890167] [v2:127.0.0.1:6804/3695890167,v1:127.0.0.1:6805/3695890167] exists,up 505cc7f4-fb42-4465-8933-5f18295caf89 2026-02-17T21:18:07.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:18:07.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:18:07.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:18:07.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:431: TEST_divergent_2: for i in $osds 2026-02-17T21:18:07.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:433: TEST_divergent_2: run_osd td/divergent-priors 1 2026-02-17T21:18:07.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-17T21:18:07.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:18:07.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T21:18:07.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:18:07.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/1 2026-02-17T21:18:07.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:18:07.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:18:07.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:18:07.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:18:07.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-17T21:18:07.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-17T21:18:07.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:18:07.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:18:07.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:18:07.670 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:18:07.670 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:18:07.670 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:18:07.670 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:18:07.670 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:18:07.670 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:18:07.670 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:18:07.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:18:07.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:18:07.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:18:07.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:18:07.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:18:07.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:18:07.671 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:18:07.671 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:18:07.671 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:18:07.671 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:18:07.671 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:18:07.671 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:18:07.671 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/1 2026-02-17T21:18:07.671 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:18:07.672 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=d8cc5e2c-83e0-49d6-8a94-49a26ec2d2cb 2026-02-17T21:18:07.672 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 d8cc5e2c-83e0-49d6-8a94-49a26ec2d2cb' 2026-02-17T21:18:07.672 INFO:tasks.workunit.client.0.trial031.stdout:add osd1 d8cc5e2c-83e0-49d6-8a94-49a26ec2d2cb 2026-02-17T21:18:07.672 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:18:07.682 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAP25RptCmJKBAABNcceQkJXj1OkL3iK/OKHA== 2026-02-17T21:18:07.682 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAP25RptCmJKBAABNcceQkJXj1OkL3iK/OKHA=="}' 2026-02-17T21:18:07.682 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new d8cc5e2c-83e0-49d6-8a94-49a26ec2d2cb -i td/divergent-priors/1/new.json 2026-02-17T21:18:07.880 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:18:07.889 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/1/new.json 2026-02-17T21:18:07.890 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAP25RptCmJKBAABNcceQkJXj1OkL3iK/OKHA== --osd-uuid d8cc5e2c-83e0-49d6-8a94-49a26ec2d2cb 2026-02-17T21:18:07.908 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:07.900+0000 7f0fef5458c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:07.910 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:07.904+0000 7f0fef5458c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:07.911 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:07.904+0000 7f0fef5458c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:07.911 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:07.904+0000 7f0fef5458c0 -1 bdev(0x55b753ffdc00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-17T21:18:07.912 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:07.904+0000 7f0fef5458c0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-17T21:18:10.170 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-17T21:18:10.170 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:18:10.171 INFO:tasks.workunit.client.0.trial031.stdout:adding osd1 key to auth repository 2026-02-17T21:18:10.171 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T21:18:10.171 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:18:10.449 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:18:10.450 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T21:18:10.450 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:18:10.450 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:18:10.450 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:18:10.450 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:18:10.478 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:10.473+0000 7f4ca840b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:10.480 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:10.473+0000 7f4ca840b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:10.481 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:10.477+0000 7f4ca840b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:10.654 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T21:18:10.654 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:18:10.654 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:18:10.654 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:18:10.654 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:18:10.654 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:10.655 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:18:10.655 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:18:10.655 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:10.655 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:18:10.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:11.857 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:11.857 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:11.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:18:11.858 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:18:11.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:11.859 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:18:12.046 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:12.041+0000 7f4ca840b8c0 -1 Falling back to public interface 2026-02-17T21:18:12.053 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:13.055 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:13.055 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:13.055 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:18:13.056 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:18:13.056 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:13.056 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:18:13.250 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:14.251 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:14.252 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:14.252 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:18:14.252 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:18:14.253 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:14.253 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:18:14.450 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:15.056 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:15.049+0000 7f4ca840b8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T21:18:15.452 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:15.452 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:15.452 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:18:15.453 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:18:15.453 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:15.453 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:18:15.647 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:16.648 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:16.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:16.649 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:18:16.650 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:18:16.650 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:16.650 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:18:16.845 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2990836993,v1:127.0.0.1:6811/2990836993] [v2:127.0.0.1:6812/2990836993,v1:127.0.0.1:6813/2990836993] exists,up d8cc5e2c-83e0-49d6-8a94-49a26ec2d2cb 2026-02-17T21:18:16.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:18:16.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:18:16.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:18:16.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:431: TEST_divergent_2: for i in $osds 2026-02-17T21:18:16.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:433: TEST_divergent_2: run_osd td/divergent-priors 2 2026-02-17T21:18:16.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-17T21:18:16.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:18:16.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T21:18:16.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:18:16.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/2 2026-02-17T21:18:16.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:18:16.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:18:16.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:18:16.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:18:16.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-17T21:18:16.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-17T21:18:16.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:18:16.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:18:16.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:18:16.850 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:18:16.850 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:18:16.850 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:18:16.851 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:18:16.851 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:18:16.851 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:18:16.851 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:18:16.851 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:18:16.851 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:18:16.851 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:18:16.851 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:18:16.851 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:18:16.851 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:18:16.851 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:18:16.851 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:18:16.851 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:18:16.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:18:16.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:18:16.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:18:16.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/2 2026-02-17T21:18:16.852 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:18:16.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=cd50e885-11ef-49a7-a859-6f1549a4e9de 2026-02-17T21:18:16.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 cd50e885-11ef-49a7-a859-6f1549a4e9de' 2026-02-17T21:18:16.853 INFO:tasks.workunit.client.0.trial031.stdout:add osd2 cd50e885-11ef-49a7-a859-6f1549a4e9de 2026-02-17T21:18:16.853 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:18:16.865 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAY25RpqVN5MxAA3TDHkKnKAPTpv1aM/Uv6Gg== 2026-02-17T21:18:16.865 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAY25RpqVN5MxAA3TDHkKnKAPTpv1aM/Uv6Gg=="}' 2026-02-17T21:18:16.865 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new cd50e885-11ef-49a7-a859-6f1549a4e9de -i td/divergent-priors/2/new.json 2026-02-17T21:18:17.066 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:18:17.075 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/2/new.json 2026-02-17T21:18:17.076 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAY25RpqVN5MxAA3TDHkKnKAPTpv1aM/Uv6Gg== --osd-uuid cd50e885-11ef-49a7-a859-6f1549a4e9de 2026-02-17T21:18:17.095 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:17.089+0000 7f5a683648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:17.096 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:17.089+0000 7f5a683648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:17.097 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:17.093+0000 7f5a683648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:17.098 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:17.093+0000 7f5a683648c0 -1 bdev(0x5638eef21c00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-17T21:18:17.098 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:17.093+0000 7f5a683648c0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-17T21:18:19.418 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-17T21:18:19.418 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:18:19.419 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T21:18:19.419 INFO:tasks.workunit.client.0.trial031.stdout:adding osd2 key to auth repository 2026-02-17T21:18:19.420 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:18:19.699 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T21:18:19.700 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:18:19.701 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:18:19.701 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:18:19.701 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:18:19.701 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:18:19.717 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:19.713+0000 7f51b03378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:19.718 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:19.713+0000 7f51b03378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:19.719 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:19.713+0000 7f51b03378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:19.903 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T21:18:19.903 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:18:19.903 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:18:19.903 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:18:19.903 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:18:19.903 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:19.904 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:18:19.904 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:18:19.904 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:19.905 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:18:20.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:21.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:21.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:21.111 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:18:21.111 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:18:21.111 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:21.112 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:18:21.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:21.462 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:21.457+0000 7f51b03378c0 -1 Falling back to public interface 2026-02-17T21:18:22.311 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:22.311 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:22.311 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:18:22.311 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:18:22.312 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:22.312 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:18:22.507 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:23.508 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:23.508 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:23.509 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:18:23.510 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:18:23.510 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:23.510 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:18:23.708 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:24.454 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:24.449+0000 7f51b03378c0 -1 osd.2 0 log_to_monitors true 2026-02-17T21:18:24.709 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:24.710 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:24.710 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:18:24.711 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:18:24.711 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:24.711 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:18:24.914 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:25.915 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:25.915 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:25.916 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:18:25.916 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:18:25.916 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:25.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:18:26.120 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2994209527,v1:127.0.0.1:6819/2994209527] [v2:127.0.0.1:6820/2994209527,v1:127.0.0.1:6821/2994209527] exists,up cd50e885-11ef-49a7-a859-6f1549a4e9de 2026-02-17T21:18:26.120 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:18:26.120 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:18:26.121 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:18:26.121 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:436: TEST_divergent_2: ceph osd set noout 2026-02-17T21:18:26.417 INFO:tasks.workunit.client.0.trial031.stderr:noout is set 2026-02-17T21:18:26.427 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:437: TEST_divergent_2: ceph osd set noin 2026-02-17T21:18:26.720 INFO:tasks.workunit.client.0.trial031.stderr:noin is set 2026-02-17T21:18:26.730 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:438: TEST_divergent_2: ceph osd set nodown 2026-02-17T21:18:27.024 INFO:tasks.workunit.client.0.trial031.stderr:nodown is set 2026-02-17T21:18:27.036 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:439: TEST_divergent_2: create_pool test 1 1 2026-02-17T21:18:27.036 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T21:18:27.274 INFO:tasks.workunit.client.0.trial031.stderr:pool 'test' already exists 2026-02-17T21:18:27.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T21:18:28.284 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:440: TEST_divergent_2: ceph osd pool set test size 3 2026-02-17T21:18:28.563 INFO:tasks.workunit.client.0.trial031.stderr:set pool 1 size to 3 2026-02-17T21:18:28.574 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:441: TEST_divergent_2: ceph osd pool set test min_size 2 2026-02-17T21:18:28.869 INFO:tasks.workunit.client.0.trial031.stderr:set pool 1 min_size to 2 2026-02-17T21:18:28.880 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:443: TEST_divergent_2: flush_pg_stats 2026-02-17T21:18:28.880 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:18:28.881 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:18:29.081 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:18:29.081 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:18:29.081 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:18:29.081 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:18:29.081 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:18:29.081 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:18:29.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T21:18:29.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T21:18:29.163 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T21:18:29.163 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:18:29.163 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:18:29.240 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-17T21:18:29.241 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-17T21:18:29.241 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669' 2026-02-17T21:18:29.241 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:18:29.241 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:18:29.319 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542147 2026-02-17T21:18:29.320 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542147 2026-02-17T21:18:29.320 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-60129542147' 2026-02-17T21:18:29.320 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:18:29.321 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T21:18:29.321 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:18:29.321 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:18:29.322 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T21:18:29.322 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:18:29.323 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T21:18:29.324 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T21:18:29.324 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 21474836487 2026-02-17T21:18:29.324 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:18:29.534 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-17T21:18:29.534 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:18:30.535 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:18:30.535 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:18:30.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T21:18:30.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:18:30.734 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-17T21:18:30.734 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:18:30.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:18:30.736 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-17T21:18:30.736 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:18:30.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-17T21:18:30.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-17T21:18:30.738 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 38654705669 2026-02-17T21:18:30.738 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:18:30.937 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-17T21:18:30.937 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:18:30.938 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542147 2026-02-17T21:18:30.938 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:18:30.939 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:18:30.940 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542147 2026-02-17T21:18:30.940 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:18:30.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542147 2026-02-17T21:18:30.942 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 60129542147 2026-02-17T21:18:30.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-17T21:18:30.942 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:18:31.147 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-17T21:18:31.148 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:444: TEST_divergent_2: wait_for_clean 2026-02-17T21:18:31.148 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:18:31.148 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:18:31.148 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:18:31.148 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T21:18:31.149 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:18:31.149 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:18:31.149 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:18:31.149 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:18:31.149 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:18:31.244 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T21:18:31.245 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:18:31.245 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:18:31.245 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:18:31.245 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:18:31.245 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:18:31.449 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:18:31.449 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:18:31.449 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:18:31.449 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:18:31.449 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:18:31.449 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:18:31.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-17T21:18:31.533 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-17T21:18:31.533 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T21:18:31.533 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:18:31.533 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:18:31.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705672 2026-02-17T21:18:31.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705672 2026-02-17T21:18:31.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705672' 2026-02-17T21:18:31.614 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:18:31.614 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:18:31.693 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542150 2026-02-17T21:18:31.693 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542150 2026-02-17T21:18:31.693 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705672 2-60129542150' 2026-02-17T21:18:31.693 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:18:31.694 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-17T21:18:31.694 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:18:31.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:18:31.695 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-17T21:18:31.695 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:18:31.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-17T21:18:31.697 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 21474836489 2026-02-17T21:18:31.697 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T21:18:31.697 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:18:31.907 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-17T21:18:31.907 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:18:32.908 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:18:32.908 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:18:33.106 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836489 2026-02-17T21:18:33.106 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:18:33.107 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705672 2026-02-17T21:18:33.107 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:18:33.107 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:18:33.108 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705672 2026-02-17T21:18:33.108 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:18:33.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705672 2026-02-17T21:18:33.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705672' 2026-02-17T21:18:33.110 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 38654705672 2026-02-17T21:18:33.111 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:18:33.323 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705672 -lt 38654705672 2026-02-17T21:18:33.323 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:18:33.324 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542150 2026-02-17T21:18:33.324 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:18:33.324 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:18:33.325 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542150 2026-02-17T21:18:33.325 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:18:33.327 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542150 2026-02-17T21:18:33.327 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-17T21:18:33.327 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 60129542150 2026-02-17T21:18:33.328 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:18:33.536 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-17T21:18:33.537 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:18:33.537 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:18:33.537 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:18:33.805 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T21:18:33.805 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:18:33.805 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:18:33.805 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:18:33.806 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:18:33.806 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:18:33.806 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:18:33.806 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:18:33.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T21:18:33.998 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:18:33.999 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:18:33.999 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:18:34.260 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T21:18:34.261 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T21:18:34.261 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T21:18:34.261 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:447: TEST_divergent_2: ceph pg dump pgs --format=json 2026-02-17T21:18:34.262 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:447: TEST_divergent_2: jq '.pg_stats[0].up_primary' 2026-02-17T21:18:34.448 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:18:34.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:447: TEST_divergent_2: local divergent=1 2026-02-17T21:18:34.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:448: TEST_divergent_2: echo 'primary and soon to be divergent is 1' 2026-02-17T21:18:34.458 INFO:tasks.workunit.client.0.trial031.stdout:primary and soon to be divergent is 1 2026-02-17T21:18:34.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:449: TEST_divergent_2: ceph pg dump pgs 2026-02-17T21:18:34.652 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:18:34.652 INFO:tasks.workunit.client.0.trial031.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T21:18:28.872767+0000 0'0 28:19 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:18:27.226024+0000 0'0 2026-02-17T21:18:27.226024+0000 0 0 periodic scrub scheduled @ 2026-02-19T01:21:29.194493+0000 0 0 2026-02-17T21:18:34.653 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:18:34.653 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:18:34.653 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:18:34.661 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:450: TEST_divergent_2: local non_divergent= 2026-02-17T21:18:34.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:451: TEST_divergent_2: for i in $osds 2026-02-17T21:18:34.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:453: TEST_divergent_2: '[' 0 = 1 ']' 2026-02-17T21:18:34.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:456: TEST_divergent_2: non_divergent=' 0' 2026-02-17T21:18:34.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:451: TEST_divergent_2: for i in $osds 2026-02-17T21:18:34.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:453: TEST_divergent_2: '[' 1 = 1 ']' 2026-02-17T21:18:34.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:454: TEST_divergent_2: continue 2026-02-17T21:18:34.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:451: TEST_divergent_2: for i in $osds 2026-02-17T21:18:34.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:453: TEST_divergent_2: '[' 2 = 1 ']' 2026-02-17T21:18:34.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:456: TEST_divergent_2: non_divergent=' 0 2' 2026-02-17T21:18:34.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:459: TEST_divergent_2: echo 'writing initial objects' 2026-02-17T21:18:34.663 INFO:tasks.workunit.client.0.trial031.stdout:writing initial objects 2026-02-17T21:18:34.663 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: seq 1 100 2026-02-17T21:18:34.663 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:34.664 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_1 /etc/fstab 2026-02-17T21:18:34.684 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:34.684 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_2 /etc/fstab 2026-02-17T21:18:34.713 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:34.714 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_3 /etc/fstab 2026-02-17T21:18:34.739 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:34.739 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_4 /etc/fstab 2026-02-17T21:18:34.767 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:34.767 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_5 /etc/fstab 2026-02-17T21:18:34.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:34.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_6 /etc/fstab 2026-02-17T21:18:34.823 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:34.823 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_7 /etc/fstab 2026-02-17T21:18:34.851 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:34.851 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_8 /etc/fstab 2026-02-17T21:18:34.879 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:34.879 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_9 /etc/fstab 2026-02-17T21:18:34.906 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:34.906 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_10 /etc/fstab 2026-02-17T21:18:34.934 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:34.934 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_11 /etc/fstab 2026-02-17T21:18:34.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:34.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_12 /etc/fstab 2026-02-17T21:18:34.989 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:34.989 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_13 /etc/fstab 2026-02-17T21:18:35.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_14 /etc/fstab 2026-02-17T21:18:35.041 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.041 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_15 /etc/fstab 2026-02-17T21:18:35.064 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.064 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_16 /etc/fstab 2026-02-17T21:18:35.089 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.089 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_17 /etc/fstab 2026-02-17T21:18:35.117 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.117 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_18 /etc/fstab 2026-02-17T21:18:35.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_19 /etc/fstab 2026-02-17T21:18:35.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_20 /etc/fstab 2026-02-17T21:18:35.201 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.201 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_21 /etc/fstab 2026-02-17T21:18:35.229 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.229 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_22 /etc/fstab 2026-02-17T21:18:35.250 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.250 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_23 /etc/fstab 2026-02-17T21:18:35.276 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.276 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_24 /etc/fstab 2026-02-17T21:18:35.300 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.301 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_25 /etc/fstab 2026-02-17T21:18:35.328 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.328 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_26 /etc/fstab 2026-02-17T21:18:35.352 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.352 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_27 /etc/fstab 2026-02-17T21:18:35.372 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.373 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_28 /etc/fstab 2026-02-17T21:18:35.398 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.398 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_29 /etc/fstab 2026-02-17T21:18:35.426 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.426 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_30 /etc/fstab 2026-02-17T21:18:35.454 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.454 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_31 /etc/fstab 2026-02-17T21:18:35.481 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.481 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_32 /etc/fstab 2026-02-17T21:18:35.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_33 /etc/fstab 2026-02-17T21:18:35.529 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.529 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_34 /etc/fstab 2026-02-17T21:18:35.555 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.556 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_35 /etc/fstab 2026-02-17T21:18:35.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_36 /etc/fstab 2026-02-17T21:18:35.605 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.605 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_37 /etc/fstab 2026-02-17T21:18:35.631 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.631 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_38 /etc/fstab 2026-02-17T21:18:35.659 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.659 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_39 /etc/fstab 2026-02-17T21:18:35.686 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.686 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_40 /etc/fstab 2026-02-17T21:18:35.714 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.714 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_41 /etc/fstab 2026-02-17T21:18:35.742 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.742 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_42 /etc/fstab 2026-02-17T21:18:35.769 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.770 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_43 /etc/fstab 2026-02-17T21:18:35.797 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.797 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_44 /etc/fstab 2026-02-17T21:18:35.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_45 /etc/fstab 2026-02-17T21:18:35.845 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.845 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_46 /etc/fstab 2026-02-17T21:18:35.871 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.872 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_47 /etc/fstab 2026-02-17T21:18:35.898 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.898 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_48 /etc/fstab 2026-02-17T21:18:35.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_49 /etc/fstab 2026-02-17T21:18:35.953 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.953 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_50 /etc/fstab 2026-02-17T21:18:35.972 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:35.972 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_51 /etc/fstab 2026-02-17T21:18:35.999 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_52 /etc/fstab 2026-02-17T21:18:36.027 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.027 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_53 /etc/fstab 2026-02-17T21:18:36.053 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.053 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_54 /etc/fstab 2026-02-17T21:18:36.081 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.081 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_55 /etc/fstab 2026-02-17T21:18:36.109 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.109 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_56 /etc/fstab 2026-02-17T21:18:36.137 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.137 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_57 /etc/fstab 2026-02-17T21:18:36.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_58 /etc/fstab 2026-02-17T21:18:36.194 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.194 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_59 /etc/fstab 2026-02-17T21:18:36.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_60 /etc/fstab 2026-02-17T21:18:36.245 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.245 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_61 /etc/fstab 2026-02-17T21:18:36.274 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.274 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_62 /etc/fstab 2026-02-17T21:18:36.302 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.302 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_63 /etc/fstab 2026-02-17T21:18:36.329 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.330 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_64 /etc/fstab 2026-02-17T21:18:36.357 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.357 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_65 /etc/fstab 2026-02-17T21:18:36.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_66 /etc/fstab 2026-02-17T21:18:36.413 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.413 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_67 /etc/fstab 2026-02-17T21:18:36.441 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.442 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_68 /etc/fstab 2026-02-17T21:18:36.469 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.470 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_69 /etc/fstab 2026-02-17T21:18:36.498 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.498 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_70 /etc/fstab 2026-02-17T21:18:36.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_71 /etc/fstab 2026-02-17T21:18:36.541 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.541 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_72 /etc/fstab 2026-02-17T21:18:36.565 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.565 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_73 /etc/fstab 2026-02-17T21:18:36.590 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.590 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_74 /etc/fstab 2026-02-17T21:18:36.612 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.612 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_75 /etc/fstab 2026-02-17T21:18:36.635 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.636 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_76 /etc/fstab 2026-02-17T21:18:36.661 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.661 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_77 /etc/fstab 2026-02-17T21:18:36.689 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.689 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_78 /etc/fstab 2026-02-17T21:18:36.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_79 /etc/fstab 2026-02-17T21:18:36.745 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.745 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_80 /etc/fstab 2026-02-17T21:18:36.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_81 /etc/fstab 2026-02-17T21:18:36.802 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.802 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_82 /etc/fstab 2026-02-17T21:18:36.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_83 /etc/fstab 2026-02-17T21:18:36.857 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_84 /etc/fstab 2026-02-17T21:18:36.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_85 /etc/fstab 2026-02-17T21:18:36.907 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.908 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_86 /etc/fstab 2026-02-17T21:18:36.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_87 /etc/fstab 2026-02-17T21:18:36.956 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.956 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_88 /etc/fstab 2026-02-17T21:18:36.982 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:36.982 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_89 /etc/fstab 2026-02-17T21:18:37.010 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:37.010 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_90 /etc/fstab 2026-02-17T21:18:37.032 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:37.032 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_91 /etc/fstab 2026-02-17T21:18:37.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:37.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_92 /etc/fstab 2026-02-17T21:18:37.088 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:37.088 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_93 /etc/fstab 2026-02-17T21:18:37.113 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:37.113 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_94 /etc/fstab 2026-02-17T21:18:37.141 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:37.141 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_95 /etc/fstab 2026-02-17T21:18:37.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:37.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_96 /etc/fstab 2026-02-17T21:18:37.196 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:37.196 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_97 /etc/fstab 2026-02-17T21:18:37.225 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:37.225 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_98 /etc/fstab 2026-02-17T21:18:37.253 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:37.254 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_99 /etc/fstab 2026-02-17T21:18:37.281 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-17T21:18:37.281 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_100 /etc/fstab 2026-02-17T21:18:37.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:466: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-17T21:18:37.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:466: TEST_divergent_2: wait_for_clean 2026-02-17T21:18:37.310 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:18:37.310 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:18:37.310 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:18:37.310 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-17T21:18:37.311 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:18:37.311 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:18:37.311 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:18:37.311 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:18:37.311 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:18:37.380 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-17T21:18:37.380 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:18:37.380 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:18:37.380 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:18:37.381 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:18:37.381 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:18:37.590 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:18:37.591 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:18:37.591 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:18:37.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:18:37.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:18:37.591 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:18:37.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836493 2026-02-17T21:18:37.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836493 2026-02-17T21:18:37.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493' 2026-02-17T21:18:37.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:18:37.670 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:18:37.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-17T21:18:37.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-17T21:18:37.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493 1-38654705675' 2026-02-17T21:18:37.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:18:37.750 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:18:37.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542153 2026-02-17T21:18:37.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542153 2026-02-17T21:18:37.830 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493 1-38654705675 2-60129542153' 2026-02-17T21:18:37.830 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:18:37.830 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836493 2026-02-17T21:18:37.830 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:18:37.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:18:37.831 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836493 2026-02-17T21:18:37.832 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:18:37.833 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836493 2026-02-17T21:18:37.833 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 21474836493 2026-02-17T21:18:37.834 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-17T21:18:37.834 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:18:38.038 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836493 2026-02-17T21:18:38.038 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:18:39.039 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:18:39.040 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:18:39.242 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836493 2026-02-17T21:18:39.242 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:18:39.243 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705675 2026-02-17T21:18:39.243 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:18:39.243 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:18:39.245 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-17T21:18:39.245 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:18:39.246 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-17T21:18:39.246 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 38654705675 2026-02-17T21:18:39.247 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-17T21:18:39.247 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:18:39.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-17T21:18:39.463 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:18:39.463 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542153 2026-02-17T21:18:39.463 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:18:39.464 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:18:39.465 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542153 2026-02-17T21:18:39.465 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:18:39.466 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542153 2026-02-17T21:18:39.467 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-17T21:18:39.467 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 60129542153 2026-02-17T21:18:39.467 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:18:39.678 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-17T21:18:39.679 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:18:39.680 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:18:39.680 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:18:39.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T21:18:39.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:18:39.942 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:18:39.942 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:18:39.942 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:18:39.942 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:18:39.942 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:18:39.942 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:18:40.141 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T21:18:40.142 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:18:40.143 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:18:40.143 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:18:40.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T21:18:40.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T21:18:40.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T21:18:40.410 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:468: TEST_divergent_2: get_pg test existing_1 2026-02-17T21:18:40.411 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-17T21:18:40.411 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=existing_1 2026-02-17T21:18:40.411 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test existing_1 2026-02-17T21:18:40.411 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T21:18:40.612 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:468: TEST_divergent_2: local pgid=1.0 2026-02-17T21:18:40.612 INFO:tasks.workunit.client.0.trial031.stdout:blackholing osds 0 2 2026-02-17T21:18:40.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:471: TEST_divergent_2: echo 'blackholing osds 0 2' 2026-02-17T21:18:40.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:472: TEST_divergent_2: ceph pg dump pgs 2026-02-17T21:18:40.806 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:18:40.806 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 0 0 0 70600 0 0 16 84 16 active+clean 2026-02-17T21:18:28.872767+0000 28'100 28:119 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:18:27.226024+0000 0'0 2026-02-17T21:18:27.226024+0000 0 0 periodic scrub scheduled @ 2026-02-19T01:21:29.194493+0000 0 0 2026-02-17T21:18:40.806 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:18:40.806 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:18:40.806 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:18:40.814 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:473: TEST_divergent_2: for i in $non_divergent 2026-02-17T21:18:40.815 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: get_asok_path osd.0 2026-02-17T21:18:40.815 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T21:18:40.815 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T21:18:40.816 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:18:40.816 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:18:40.816 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:18:40.816 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.0.asok 2026-02-17T21:18:40.817 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: CEPH_ARGS= 2026-02-17T21:18:40.817 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-17T21:18:40.869 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:18:40.869 INFO:tasks.workunit.client.0.trial031.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-17T21:18:40.869 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:18:40.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:473: TEST_divergent_2: for i in $non_divergent 2026-02-17T21:18:40.875 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: get_asok_path osd.2 2026-02-17T21:18:40.876 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-17T21:18:40.876 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-17T21:18:40.876 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:18:40.876 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:18:40.876 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:18:40.876 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.2.asok 2026-02-17T21:18:40.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: CEPH_ARGS= 2026-02-17T21:18:40.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-17T21:18:40.929 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:18:40.929 INFO:tasks.workunit.client.0.trial031.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-17T21:18:40.929 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:18:40.934 INFO:tasks.workunit.client.0.trial031.stdout:create new divergent objects 2026-02-17T21:18:40.935 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:479: TEST_divergent_2: echo 'create new divergent objects' 2026-02-17T21:18:40.935 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:480: TEST_divergent_2: seq 1 2 2026-02-17T21:18:40.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:480: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-17T21:18:40.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:480: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-17T21:18:40.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:482: TEST_divergent_2: rados -p test create newobject_1 2026-02-17T21:18:40.936 INFO:tasks.workunit.client.0.trial031.stdout:writing divergent objects 2026-02-17T21:18:40.937 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:485: TEST_divergent_2: echo 'writing divergent objects' 2026-02-17T21:18:40.937 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:482: TEST_divergent_2: rados -p test create newobject_2 2026-02-17T21:18:40.937 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:486: TEST_divergent_2: seq 1 2 2026-02-17T21:18:40.938 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:486: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-17T21:18:40.938 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:486: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-17T21:18:40.938 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:488: TEST_divergent_2: rados -p test put existing_1 /etc/resolv.conf 2026-02-17T21:18:40.939 INFO:tasks.workunit.client.0.trial031.stdout:remove divergent objects 2026-02-17T21:18:40.939 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:491: TEST_divergent_2: echo 'remove divergent objects' 2026-02-17T21:18:40.939 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:488: TEST_divergent_2: rados -p test put existing_2 /etc/resolv.conf 2026-02-17T21:18:40.940 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:492: TEST_divergent_2: seq 1 2 2026-02-17T21:18:40.940 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:492: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-17T21:18:40.940 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: expr 1 + 2 2026-02-17T21:18:40.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: rmi=3 2026-02-17T21:18:40.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:492: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-17T21:18:40.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: rados -p test rm existing_3 2026-02-17T21:18:40.942 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: expr 2 + 2 2026-02-17T21:18:40.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: rmi=4 2026-02-17T21:18:40.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:497: TEST_divergent_2: sleep 10 2026-02-17T21:18:40.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: rados -p test rm existing_4 2026-02-17T21:18:50.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:498: TEST_divergent_2: killall -9 rados 2026-02-17T21:18:50.991 INFO:tasks.workunit.client.0.trial031.stderr:rados: no process found 2026-02-17T21:18:50.991 INFO:tasks.workunit.client.0.trial031.stdout:killing all the osds 2026-02-17T21:18:50.992 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:501: TEST_divergent_2: echo 'killing all the osds' 2026-02-17T21:18:50.992 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:502: TEST_divergent_2: ceph pg dump pgs 2026-02-17T21:18:51.200 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:18:51.200 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 0 0 0 69710 0 0 16 90 16 active+clean 2026-02-17T21:18:28.872767+0000 28'106 28:125 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:18:27.226024+0000 0'0 2026-02-17T21:18:27.226024+0000 0 0 periodic scrub scheduled @ 2026-02-19T01:21:29.194493+0000 0 0 2026-02-17T21:18:51.200 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:18:51.200 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:18:51.200 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:18:51.210 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:503: TEST_divergent_2: kill_daemons td/divergent-priors KILL osd 2026-02-17T21:18:51.210 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:18:51.211 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:18:51.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:18:51.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:18:51.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:18:51.319 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:18:51.319 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:504: TEST_divergent_2: for i in $osds 2026-02-17T21:18:51.319 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:506: TEST_divergent_2: ceph osd down osd.0 2026-02-17T21:18:51.577 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already down. 2026-02-17T21:18:51.587 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:504: TEST_divergent_2: for i in $osds 2026-02-17T21:18:51.587 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:506: TEST_divergent_2: ceph osd down osd.1 2026-02-17T21:18:51.855 INFO:tasks.workunit.client.0.trial031.stderr:osd.1 is already down. 2026-02-17T21:18:51.864 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:504: TEST_divergent_2: for i in $osds 2026-02-17T21:18:51.864 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:506: TEST_divergent_2: ceph osd down osd.2 2026-02-17T21:18:52.159 INFO:tasks.workunit.client.0.trial031.stderr:osd.2 is already down. 2026-02-17T21:18:52.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:508: TEST_divergent_2: for i in $non_divergent 2026-02-17T21:18:52.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:510: TEST_divergent_2: ceph osd out osd.0 2026-02-17T21:18:52.435 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already out. 2026-02-17T21:18:52.445 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:508: TEST_divergent_2: for i in $non_divergent 2026-02-17T21:18:52.445 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:510: TEST_divergent_2: ceph osd out osd.2 2026-02-17T21:18:52.713 INFO:tasks.workunit.client.0.trial031.stderr:osd.2 is already out. 2026-02-17T21:18:52.722 INFO:tasks.workunit.client.0.trial031.stdout:bringing up non_divergent 0 2 2026-02-17T21:18:52.723 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:514: TEST_divergent_2: echo 'bringing up non_divergent 0 2' 2026-02-17T21:18:52.723 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:515: TEST_divergent_2: ceph pg dump pgs 2026-02-17T21:18:52.930 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:18:52.930 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 0 0 0 69710 0 0 16 90 16 stale+active+clean 2026-02-17T21:18:28.872767+0000 28'106 28:125 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:18:27.226024+0000 0'0 2026-02-17T21:18:27.226024+0000 0 0 periodic scrub scheduled @ 2026-02-19T01:21:29.194493+0000 0 0 2026-02-17T21:18:52.930 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:18:52.930 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:18:52.931 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:18:52.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:516: TEST_divergent_2: for i in $non_divergent 2026-02-17T21:18:52.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:518: TEST_divergent_2: activate_osd td/divergent-priors 0 2026-02-17T21:18:52.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:18:52.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:18:52.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T21:18:52.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:18:52.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/0 2026-02-17T21:18:52.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:18:52.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:18:52.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:18:52.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:18:52.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-17T21:18:52.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-17T21:18:52.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:18:52.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:18:52.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:18:52.943 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:18:52.943 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:18:52.943 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:18:52.943 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:18:52.944 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:18:52.944 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:18:52.944 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:18:52.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:18:52.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:18:52.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:18:52.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:18:52.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:18:52.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:18:52.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:18:52.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:18:52.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:18:52.945 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:18:52.945 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/0 2026-02-17T21:18:52.945 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:18:52.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T21:18:52.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:18:52.946 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/0/whoami 2026-02-17T21:18:52.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T21:18:52.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:18:52.947 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:18:52.947 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:18:52.963 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:52.961+0000 7f053f0e98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:52.965 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:52.961+0000 7f053f0e98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:52.966 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:52.961+0000 7f053f0e98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:53.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T21:18:53.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:18:53.146 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:18:53.146 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:18:53.146 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:18:53.146 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:53.146 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:18:53.147 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:18:53.147 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:53.147 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:18:53.347 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:54.348 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:54.348 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:54.349 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:18:54.349 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:18:54.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:54.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:18:54.461 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:54.457+0000 7f053f0e98c0 -1 Falling back to public interface 2026-02-17T21:18:54.555 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:55.557 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:55.557 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:55.557 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:18:55.558 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:18:55.558 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:55.558 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:18:55.759 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:56.760 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:56.760 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:56.761 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:18:56.761 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:18:56.762 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:56.762 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:18:56.960 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:57.493 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:57.489+0000 7f053f0e98c0 -1 osd.0 28 log_to_monitors true 2026-02-17T21:18:57.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:57.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:57.962 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:18:57.962 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:18:57.962 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:57.963 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:18:58.169 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up out weight 0 up_from 35 up_thru 0 down_at 29 last_clean_interval [5,28) [v2:127.0.0.1:6802/1153784009,v1:127.0.0.1:6803/1153784009] [v2:127.0.0.1:6804/1153784009,v1:127.0.0.1:6805/1153784009] exists,up 505cc7f4-fb42-4465-8933-5f18295caf89 2026-02-17T21:18:58.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:18:58.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:18:58.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:18:58.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:516: TEST_divergent_2: for i in $non_divergent 2026-02-17T21:18:58.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:518: TEST_divergent_2: activate_osd td/divergent-priors 2 2026-02-17T21:18:58.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:18:58.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:18:58.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T21:18:58.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:18:58.174 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/2 2026-02-17T21:18:58.174 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:18:58.174 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:18:58.174 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:18:58.174 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:18:58.174 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-17T21:18:58.174 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-17T21:18:58.174 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:18:58.174 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:18:58.174 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:18:58.174 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:18:58.174 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:18:58.175 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:18:58.175 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:18:58.175 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:18:58.175 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:18:58.175 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:18:58.175 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:18:58.175 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:18:58.175 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:18:58.175 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:18:58.175 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:18:58.175 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:18:58.175 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:18:58.175 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:18:58.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:18:58.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:18:58.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/2 2026-02-17T21:18:58.176 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:18:58.177 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T21:18:58.177 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:18:58.177 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/2/whoami 2026-02-17T21:18:58.177 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T21:18:58.177 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:18:58.177 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:18:58.178 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:18:58.199 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:58.193+0000 7fac929f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:58.200 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:58.197+0000 7fac929f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:58.201 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:58.197+0000 7fac929f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:18:58.374 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T21:18:58.374 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:18:58.374 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:18:58.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:18:58.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:18:58.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:58.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:18:58.375 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:18:58.376 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:58.376 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:18:58.582 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:18:59.481 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:18:59.477+0000 7fac929f38c0 -1 Falling back to public interface 2026-02-17T21:18:59.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:18:59.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:18:59.584 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:18:59.585 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:18:59.585 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:18:59.585 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:18:59.789 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:19:00.791 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:19:00.791 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:19:00.791 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:19:00.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:19:00.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:19:00.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:19:00.997 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:19:01.999 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:19:01.999 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:19:01.999 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:19:01.999 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:19:02.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:19:02.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:19:02.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:19:02.746 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:19:02.741+0000 7fac929f38c0 -1 osd.2 28 log_to_monitors true 2026-02-17T21:19:03.208 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:19:03.208 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:19:03.209 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:19:03.209 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:19:03.210 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:19:03.210 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:19:03.409 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up out weight 0 up_from 38 up_thru 0 down_at 31 last_clean_interval [14,28) [v2:127.0.0.1:6810/2603940079,v1:127.0.0.1:6811/2603940079] [v2:127.0.0.1:6812/2603940079,v1:127.0.0.1:6813/2603940079] exists,up cd50e885-11ef-49a7-a859-6f1549a4e9de 2026-02-17T21:19:03.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:19:03.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:19:03.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:19:03.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:520: TEST_divergent_2: for i in $non_divergent 2026-02-17T21:19:03.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:522: TEST_divergent_2: ceph osd in osd.0 2026-02-17T21:19:03.659 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already in. 2026-02-17T21:19:03.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:520: TEST_divergent_2: for i in $non_divergent 2026-02-17T21:19:03.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:522: TEST_divergent_2: ceph osd in osd.2 2026-02-17T21:19:03.967 INFO:tasks.workunit.client.0.trial031.stderr:osd.2 is already in. 2026-02-17T21:19:03.978 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:525: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-17T21:19:03.978 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:525: TEST_divergent_2: wait_for_clean 2026-02-17T21:19:03.978 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:19:03.978 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:19:03.978 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:19:03.978 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-17T21:19:03.979 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:19:03.980 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:19:03.980 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:19:03.980 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:19:03.980 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:19:04.046 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-17T21:19:04.047 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:19:04.047 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:19:04.047 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:19:04.047 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:19:04.047 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:19:04.254 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:19:04.254 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:19:04.254 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:19:04.254 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:19:04.255 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:19:04.255 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:19:04.333 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855364 2026-02-17T21:19:04.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855364 2026-02-17T21:19:04.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855364' 2026-02-17T21:19:04.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:19:04.334 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:19:04.394 INFO:tasks.workunit.client.0.trial031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-17T21:19:04.397 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-17T21:19:04.397 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-17T21:19:04.397 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-17T21:19:04.397 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:19:04.398 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:19:04.476 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757251 2026-02-17T21:19:04.477 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757251 2026-02-17T21:19:04.477 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855364 2-163208757251' 2026-02-17T21:19:04.477 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:19:04.477 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855364 2026-02-17T21:19:04.478 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:19:04.478 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:19:04.479 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855364 2026-02-17T21:19:04.479 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:19:04.480 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855364 2026-02-17T21:19:04.480 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855364' 2026-02-17T21:19:04.481 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 150323855364 2026-02-17T21:19:04.481 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:19:04.691 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855362 -lt 150323855364 2026-02-17T21:19:04.691 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:19:05.692 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:19:05.692 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:19:05.900 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855362 -lt 150323855364 2026-02-17T21:19:05.900 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:19:06.901 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T21:19:06.902 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:19:07.103 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855364 -lt 150323855364 2026-02-17T21:19:07.104 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:19:07.104 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-163208757251 2026-02-17T21:19:07.104 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:19:07.105 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:19:07.106 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-163208757251 2026-02-17T21:19:07.106 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:19:07.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757251 2026-02-17T21:19:07.108 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 163208757251 2026-02-17T21:19:07.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 163208757251' 2026-02-17T21:19:07.109 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:19:07.318 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757251 -lt 163208757251 2026-02-17T21:19:07.319 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:19:07.320 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:07.320 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:07.588 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T21:19:07.588 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:07.588 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:07.588 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:07.588 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:07.588 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:07.588 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:07.589 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:07.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:07.795 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:07.795 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:07.796 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:08.066 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:08.067 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T21:19:08.067 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T21:19:08.067 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T21:19:08.067 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:08.067 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T21:19:08.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:19:08.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:08.169 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:08.169 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:08.169 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:08.169 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:08.169 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:08.169 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:08.371 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:08.372 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:08.372 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:08.372 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:08.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:08.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:19:08.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:19:08.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:19:08.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:19:08.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:19:08.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:19:08.644 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:19:08.645 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:19:08.905 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:19:08.905 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:19:08.906 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-17T21:19:08.906 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:08.906 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T21:19:09.106 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:19:09.106 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:09.107 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:09.108 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:09.108 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:09.108 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:09.108 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:09.108 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:09.305 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:09.306 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:09.306 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:09.306 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:09.565 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:09.565 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:19:09.566 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:19:09.566 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:19:09.566 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:19:09.566 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:19:09.566 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:19:09.566 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:19:09.566 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:19:09.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:19:09.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:19:09.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-17T21:19:09.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:09.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T21:19:10.225 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:19:10.225 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:10.226 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:10.226 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:10.226 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:10.226 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:10.226 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:10.226 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:10.423 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:10.424 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:10.424 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:10.424 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:10.688 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:10.688 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:19:10.688 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:19:10.688 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:19:10.688 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:19:10.688 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:19:10.688 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:19:10.688 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:19:10.689 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:19:10.955 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:19:10.955 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:19:10.955 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-17T21:19:10.955 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:10.955 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T21:19:11.755 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:19:11.756 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:11.757 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:11.757 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:11.757 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:11.757 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:11.757 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:11.757 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:11.951 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:11.951 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:11.952 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:11.952 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:12.224 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:12.224 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:19:12.224 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:19:12.225 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:19:12.225 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:19:12.225 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:19:12.225 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:19:12.225 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:19:12.225 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:19:12.491 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:19:12.491 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:19:12.492 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-17T21:19:12.492 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:12.492 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-17T21:19:14.092 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:19:14.092 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:14.093 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:14.093 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:14.093 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:14.093 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:14.094 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:14.094 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:14.296 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:14.296 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:14.297 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:14.297 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:14.564 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:14.564 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:19:14.564 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:19:14.564 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:19:14.564 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:19:14.564 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:19:14.564 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:19:14.564 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:19:14.564 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:19:14.833 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:19:14.833 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:19:14.833 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-17T21:19:14.833 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:14.834 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-17T21:19:18.034 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:19:18.034 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:18.034 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:18.034 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:18.034 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:18.035 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:18.035 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:18.035 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:18.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:18.238 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:18.238 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:18.239 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:18.504 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:18.504 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:19:18.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:19:18.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:19:18.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:19:18.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:19:18.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:19:18.505 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:19:18.505 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:19:18.771 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:19:18.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:19:18.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-17T21:19:18.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:18.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-17T21:19:25.172 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:19:25.172 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:25.173 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:25.173 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:25.173 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:25.173 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:25.174 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:25.174 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:25.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:25.376 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:25.376 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:25.376 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:25.643 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:25.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:19:25.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:19:25.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:19:25.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:19:25.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:19:25.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:19:25.644 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:19:25.644 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:19:25.910 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:19:25.911 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:19:25.911 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-17T21:19:25.911 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:25.911 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-17T21:19:33.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:19:33.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:33.212 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:33.212 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:33.212 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:33.212 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:33.213 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:33.213 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:33.413 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:33.413 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:33.414 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:33.414 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:33.680 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:33.680 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:19:33.680 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:19:33.680 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:19:33.680 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:19:33.680 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:19:33.680 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:19:33.681 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:19:33.681 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:19:33.945 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:19:33.945 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:19:33.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-17T21:19:33.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-17T21:19:34.204 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:19:34.204 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_fingerprint": "06112e0a-c553-46df-b7de-3091abea00ce", 2026-02-17T21:19:34.205 INFO:tasks.workunit.client.0.trial031.stdout: "version": "19.2.3-679-gd855f53b", 2026-02-17T21:19:34.205 INFO:tasks.workunit.client.0.trial031.stdout: "commit": "d855f53b89fdcec760fd9232a5fb55ed4fb111a1", 2026-02-17T21:19:34.205 INFO:tasks.workunit.client.0.trial031.stdout: "timestamp": "2026-02-17T21:19:34.204865+0000", 2026-02-17T21:19:34.205 INFO:tasks.workunit.client.0.trial031.stdout: "tag": "", 2026-02-17T21:19:34.205 INFO:tasks.workunit.client.0.trial031.stdout: "health": { 2026-02-17T21:19:34.205 INFO:tasks.workunit.client.0.trial031.stdout: "status": "HEALTH_WARN", 2026-02-17T21:19:34.205 INFO:tasks.workunit.client.0.trial031.stdout: "checks": { 2026-02-17T21:19:34.205 INFO:tasks.workunit.client.0.trial031.stdout: "OSDMAP_FLAGS": { 2026-02-17T21:19:34.205 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:19:34.205 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:19:34.205 INFO:tasks.workunit.client.0.trial031.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-17T21:19:34.205 INFO:tasks.workunit.client.0.trial031.stdout: "count": 17 2026-02-17T21:19:34.206 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.206 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [], 2026-02-17T21:19:34.206 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:19:34.206 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.206 INFO:tasks.workunit.client.0.trial031.stdout: "OSD_DOWN": { 2026-02-17T21:19:34.206 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:19:34.206 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:19:34.206 INFO:tasks.workunit.client.0.trial031.stdout: "message": "1 osds down", 2026-02-17T21:19:34.206 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:19:34.206 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.206 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:19:34.206 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.207 INFO:tasks.workunit.client.0.trial031.stdout: "message": "osd.1 (root=default,host=trial031) is down" 2026-02-17T21:19:34.207 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.207 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.207 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:19:34.207 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.207 INFO:tasks.workunit.client.0.trial031.stdout: "PG_DEGRADED": { 2026-02-17T21:19:34.207 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:19:34.207 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:19:34.207 INFO:tasks.workunit.client.0.trial031.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2026-02-17T21:19:34.207 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:19:34.207 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.207 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:19:34.208 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.208 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:19:34.208 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.208 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.208 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:19:34.208 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.208 INFO:tasks.workunit.client.0.trial031.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-17T21:19:34.208 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:19:34.208 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:19:34.208 INFO:tasks.workunit.client.0.trial031.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-17T21:19:34.208 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:19:34.208 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.208 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:19:34.209 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.209 INFO:tasks.workunit.client.0.trial031.stdout: "message": "application not enabled on pool 'test'" 2026-02-17T21:19:34.209 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.209 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.209 INFO:tasks.workunit.client.0.trial031.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-17T21:19:34.209 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.209 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.209 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:19:34.209 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.209 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.209 INFO:tasks.workunit.client.0.trial031.stdout: "mutes": [] 2026-02-17T21:19:34.209 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.209 INFO:tasks.workunit.client.0.trial031.stdout: "monmap_first_committed": 1, 2026-02-17T21:19:34.210 INFO:tasks.workunit.client.0.trial031.stdout: "monmap_last_committed": 1, 2026-02-17T21:19:34.210 INFO:tasks.workunit.client.0.trial031.stdout: "monmap": { 2026-02-17T21:19:34.210 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 1, 2026-02-17T21:19:34.210 INFO:tasks.workunit.client.0.trial031.stdout: "fsid": "5ebac6e7-5321-4893-ad68-c06127ed455b", 2026-02-17T21:19:34.210 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:17:57.669569Z", 2026-02-17T21:19:34.210 INFO:tasks.workunit.client.0.trial031.stdout: "created": "2026-02-17T21:17:57.669569Z", 2026-02-17T21:19:34.210 INFO:tasks.workunit.client.0.trial031.stdout: "min_mon_release": 19, 2026-02-17T21:19:34.210 INFO:tasks.workunit.client.0.trial031.stdout: "min_mon_release_name": "squid", 2026-02-17T21:19:34.210 INFO:tasks.workunit.client.0.trial031.stdout: "election_strategy": 1, 2026-02-17T21:19:34.210 INFO:tasks.workunit.client.0.trial031.stdout: "disallowed_leaders": "", 2026-02-17T21:19:34.210 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode": false, 2026-02-17T21:19:34.210 INFO:tasks.workunit.client.0.trial031.stdout: "tiebreaker_mon": "", 2026-02-17T21:19:34.210 INFO:tasks.workunit.client.0.trial031.stdout: "removed_ranks": "", 2026-02-17T21:19:34.211 INFO:tasks.workunit.client.0.trial031.stdout: "features": { 2026-02-17T21:19:34.211 INFO:tasks.workunit.client.0.trial031.stdout: "persistent": [ 2026-02-17T21:19:34.211 INFO:tasks.workunit.client.0.trial031.stdout: "kraken", 2026-02-17T21:19:34.211 INFO:tasks.workunit.client.0.trial031.stdout: "luminous", 2026-02-17T21:19:34.211 INFO:tasks.workunit.client.0.trial031.stdout: "mimic", 2026-02-17T21:19:34.211 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap-prune", 2026-02-17T21:19:34.211 INFO:tasks.workunit.client.0.trial031.stdout: "nautilus", 2026-02-17T21:19:34.211 INFO:tasks.workunit.client.0.trial031.stdout: "octopus", 2026-02-17T21:19:34.211 INFO:tasks.workunit.client.0.trial031.stdout: "pacific", 2026-02-17T21:19:34.211 INFO:tasks.workunit.client.0.trial031.stdout: "elector-pinging", 2026-02-17T21:19:34.211 INFO:tasks.workunit.client.0.trial031.stdout: "quincy", 2026-02-17T21:19:34.211 INFO:tasks.workunit.client.0.trial031.stdout: "reef", 2026-02-17T21:19:34.211 INFO:tasks.workunit.client.0.trial031.stdout: "squid" 2026-02-17T21:19:34.212 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.212 INFO:tasks.workunit.client.0.trial031.stdout: "optional": [] 2026-02-17T21:19:34.212 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.212 INFO:tasks.workunit.client.0.trial031.stdout: "mons": [ 2026-02-17T21:19:34.212 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.212 INFO:tasks.workunit.client.0.trial031.stdout: "rank": 0, 2026-02-17T21:19:34.212 INFO:tasks.workunit.client.0.trial031.stdout: "name": "a", 2026-02-17T21:19:34.212 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:19:34.212 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:19:34.212 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.212 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:19:34.212 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:7115", 2026-02-17T21:19:34.212 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 0 2026-02-17T21:19:34.212 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.213 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.213 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.213 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:7115/0", 2026-02-17T21:19:34.213 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-17T21:19:34.213 INFO:tasks.workunit.client.0.trial031.stdout: "priority": 0, 2026-02-17T21:19:34.213 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 0, 2026-02-17T21:19:34.213 INFO:tasks.workunit.client.0.trial031.stdout: "crush_location": "{}" 2026-02-17T21:19:34.213 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.213 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.213 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.213 INFO:tasks.workunit.client.0.trial031.stdout: "quorum": [ 2026-02-17T21:19:34.213 INFO:tasks.workunit.client.0.trial031.stdout: 0 2026-02-17T21:19:34.213 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.214 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap": { 2026-02-17T21:19:34.214 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 43, 2026-02-17T21:19:34.214 INFO:tasks.workunit.client.0.trial031.stdout: "fsid": "5ebac6e7-5321-4893-ad68-c06127ed455b", 2026-02-17T21:19:34.214 INFO:tasks.workunit.client.0.trial031.stdout: "created": "2026-02-17T21:17:57.753415+0000", 2026-02-17T21:19:34.214 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:19:04.016405+0000", 2026-02-17T21:19:34.214 INFO:tasks.workunit.client.0.trial031.stdout: "last_up_change": "2026-02-17T21:19:02.902098+0000", 2026-02-17T21:19:34.214 INFO:tasks.workunit.client.0.trial031.stdout: "last_in_change": "2026-02-17T21:19:03.915798+0000", 2026-02-17T21:19:34.214 INFO:tasks.workunit.client.0.trial031.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-17T21:19:34.214 INFO:tasks.workunit.client.0.trial031.stdout: "flags_num": 5800384, 2026-02-17T21:19:34.214 INFO:tasks.workunit.client.0.trial031.stdout: "flags_set": [ 2026-02-17T21:19:34.214 INFO:tasks.workunit.client.0.trial031.stdout: "nodown", 2026-02-17T21:19:34.214 INFO:tasks.workunit.client.0.trial031.stdout: "noin", 2026-02-17T21:19:34.215 INFO:tasks.workunit.client.0.trial031.stdout: "noout", 2026-02-17T21:19:34.215 INFO:tasks.workunit.client.0.trial031.stdout: "pglog_hardlimit", 2026-02-17T21:19:34.215 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snapdirs", 2026-02-17T21:19:34.215 INFO:tasks.workunit.client.0.trial031.stdout: "recovery_deletes", 2026-02-17T21:19:34.215 INFO:tasks.workunit.client.0.trial031.stdout: "sortbitwise" 2026-02-17T21:19:34.215 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.215 INFO:tasks.workunit.client.0.trial031.stdout: "crush_version": 9, 2026-02-17T21:19:34.215 INFO:tasks.workunit.client.0.trial031.stdout: "full_ratio": 0.99000000953674316, 2026-02-17T21:19:34.215 INFO:tasks.workunit.client.0.trial031.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-17T21:19:34.215 INFO:tasks.workunit.client.0.trial031.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-17T21:19:34.215 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_snapshot": "", 2026-02-17T21:19:34.215 INFO:tasks.workunit.client.0.trial031.stdout: "pool_max": 1, 2026-02-17T21:19:34.216 INFO:tasks.workunit.client.0.trial031.stdout: "max_osd": 3, 2026-02-17T21:19:34.216 INFO:tasks.workunit.client.0.trial031.stdout: "require_min_compat_client": "luminous", 2026-02-17T21:19:34.216 INFO:tasks.workunit.client.0.trial031.stdout: "min_compat_client": "jewel", 2026-02-17T21:19:34.216 INFO:tasks.workunit.client.0.trial031.stdout: "require_osd_release": "squid", 2026-02-17T21:19:34.216 INFO:tasks.workunit.client.0.trial031.stdout: "allow_crimson": false, 2026-02-17T21:19:34.216 INFO:tasks.workunit.client.0.trial031.stdout: "pools": [ 2026-02-17T21:19:34.216 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.216 INFO:tasks.workunit.client.0.trial031.stdout: "pool": 1, 2026-02-17T21:19:34.216 INFO:tasks.workunit.client.0.trial031.stdout: "pool_name": "test", 2026-02-17T21:19:34.216 INFO:tasks.workunit.client.0.trial031.stdout: "create_time": "2026-02-17T21:18:27.190988+0000", 2026-02-17T21:19:34.216 INFO:tasks.workunit.client.0.trial031.stdout: "flags": 1, 2026-02-17T21:19:34.216 INFO:tasks.workunit.client.0.trial031.stdout: "flags_names": "hashpspool", 2026-02-17T21:19:34.216 INFO:tasks.workunit.client.0.trial031.stdout: "type": 1, 2026-02-17T21:19:34.217 INFO:tasks.workunit.client.0.trial031.stdout: "size": 3, 2026-02-17T21:19:34.217 INFO:tasks.workunit.client.0.trial031.stdout: "min_size": 2, 2026-02-17T21:19:34.217 INFO:tasks.workunit.client.0.trial031.stdout: "crush_rule": 0, 2026-02-17T21:19:34.217 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_count": 0, 2026-02-17T21:19:34.217 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_target": 0, 2026-02-17T21:19:34.217 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_barrier": 0, 2026-02-17T21:19:34.217 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-17T21:19:34.217 INFO:tasks.workunit.client.0.trial031.stdout: "is_stretch_pool": false, 2026-02-17T21:19:34.217 INFO:tasks.workunit.client.0.trial031.stdout: "object_hash": 2, 2026-02-17T21:19:34.217 INFO:tasks.workunit.client.0.trial031.stdout: "pg_autoscale_mode": "off", 2026-02-17T21:19:34.217 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num": 1, 2026-02-17T21:19:34.217 INFO:tasks.workunit.client.0.trial031.stdout: "pg_placement_num": 1, 2026-02-17T21:19:34.217 INFO:tasks.workunit.client.0.trial031.stdout: "pg_placement_num_target": 1, 2026-02-17T21:19:34.218 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num_target": 1, 2026-02-17T21:19:34.218 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num_pending": 1, 2026-02-17T21:19:34.218 INFO:tasks.workunit.client.0.trial031.stdout: "last_pg_merge_meta": { 2026-02-17T21:19:34.218 INFO:tasks.workunit.client.0.trial031.stdout: "source_pgid": "0.0", 2026-02-17T21:19:34.218 INFO:tasks.workunit.client.0.trial031.stdout: "ready_epoch": 0, 2026-02-17T21:19:34.218 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_started": 0, 2026-02-17T21:19:34.218 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_clean": 0, 2026-02-17T21:19:34.218 INFO:tasks.workunit.client.0.trial031.stdout: "source_version": "0'0", 2026-02-17T21:19:34.218 INFO:tasks.workunit.client.0.trial031.stdout: "target_version": "0'0" 2026-02-17T21:19:34.218 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.218 INFO:tasks.workunit.client.0.trial031.stdout: "last_change": "28", 2026-02-17T21:19:34.218 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend": "0", 2026-02-17T21:19:34.219 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-17T21:19:34.219 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend_preluminous": "0", 2026-02-17T21:19:34.219 INFO:tasks.workunit.client.0.trial031.stdout: "auid": 0, 2026-02-17T21:19:34.219 INFO:tasks.workunit.client.0.trial031.stdout: "snap_mode": "selfmanaged", 2026-02-17T21:19:34.219 INFO:tasks.workunit.client.0.trial031.stdout: "snap_seq": 0, 2026-02-17T21:19:34.219 INFO:tasks.workunit.client.0.trial031.stdout: "snap_epoch": 0, 2026-02-17T21:19:34.219 INFO:tasks.workunit.client.0.trial031.stdout: "pool_snaps": [], 2026-02-17T21:19:34.219 INFO:tasks.workunit.client.0.trial031.stdout: "removed_snaps": "[]", 2026-02-17T21:19:34.219 INFO:tasks.workunit.client.0.trial031.stdout: "quota_max_bytes": 0, 2026-02-17T21:19:34.219 INFO:tasks.workunit.client.0.trial031.stdout: "quota_max_objects": 0, 2026-02-17T21:19:34.219 INFO:tasks.workunit.client.0.trial031.stdout: "tiers": [], 2026-02-17T21:19:34.219 INFO:tasks.workunit.client.0.trial031.stdout: "tier_of": -1, 2026-02-17T21:19:34.220 INFO:tasks.workunit.client.0.trial031.stdout: "read_tier": -1, 2026-02-17T21:19:34.220 INFO:tasks.workunit.client.0.trial031.stdout: "write_tier": -1, 2026-02-17T21:19:34.220 INFO:tasks.workunit.client.0.trial031.stdout: "cache_mode": "none", 2026-02-17T21:19:34.220 INFO:tasks.workunit.client.0.trial031.stdout: "target_max_bytes": 0, 2026-02-17T21:19:34.220 INFO:tasks.workunit.client.0.trial031.stdout: "target_max_objects": 0, 2026-02-17T21:19:34.220 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-17T21:19:34.220 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-17T21:19:34.220 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-17T21:19:34.220 INFO:tasks.workunit.client.0.trial031.stdout: "cache_min_flush_age": 0, 2026-02-17T21:19:34.220 INFO:tasks.workunit.client.0.trial031.stdout: "cache_min_evict_age": 0, 2026-02-17T21:19:34.220 INFO:tasks.workunit.client.0.trial031.stdout: "erasure_code_profile": "", 2026-02-17T21:19:34.220 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_params": { 2026-02-17T21:19:34.221 INFO:tasks.workunit.client.0.trial031.stdout: "type": "none" 2026-02-17T21:19:34.221 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.221 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_period": 0, 2026-02-17T21:19:34.221 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_count": 0, 2026-02-17T21:19:34.221 INFO:tasks.workunit.client.0.trial031.stdout: "use_gmt_hitset": true, 2026-02-17T21:19:34.221 INFO:tasks.workunit.client.0.trial031.stdout: "min_read_recency_for_promote": 0, 2026-02-17T21:19:34.221 INFO:tasks.workunit.client.0.trial031.stdout: "min_write_recency_for_promote": 0, 2026-02-17T21:19:34.221 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_grade_decay_rate": 0, 2026-02-17T21:19:34.221 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_search_last_n": 0, 2026-02-17T21:19:34.221 INFO:tasks.workunit.client.0.trial031.stdout: "grade_table": [], 2026-02-17T21:19:34.221 INFO:tasks.workunit.client.0.trial031.stdout: "stripe_width": 0, 2026-02-17T21:19:34.221 INFO:tasks.workunit.client.0.trial031.stdout: "expected_num_objects": 0, 2026-02-17T21:19:34.221 INFO:tasks.workunit.client.0.trial031.stdout: "fast_read": false, 2026-02-17T21:19:34.222 INFO:tasks.workunit.client.0.trial031.stdout: "options": {}, 2026-02-17T21:19:34.222 INFO:tasks.workunit.client.0.trial031.stdout: "application_metadata": {}, 2026-02-17T21:19:34.222 INFO:tasks.workunit.client.0.trial031.stdout: "read_balance": { 2026-02-17T21:19:34.222 INFO:tasks.workunit.client.0.trial031.stdout: "score_type": "Fair distribution", 2026-02-17T21:19:34.222 INFO:tasks.workunit.client.0.trial031.stdout: "score_acting": 2.9900000095367432, 2026-02-17T21:19:34.222 INFO:tasks.workunit.client.0.trial031.stdout: "score_stable": 2.9900000095367432, 2026-02-17T21:19:34.222 INFO:tasks.workunit.client.0.trial031.stdout: "optimal_score": 0.67000001668930054, 2026-02-17T21:19:34.222 INFO:tasks.workunit.client.0.trial031.stdout: "raw_score_acting": 2, 2026-02-17T21:19:34.222 INFO:tasks.workunit.client.0.trial031.stdout: "raw_score_stable": 2, 2026-02-17T21:19:34.222 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity_weighted": 1, 2026-02-17T21:19:34.222 INFO:tasks.workunit.client.0.trial031.stdout: "average_primary_affinity": 1, 2026-02-17T21:19:34.222 INFO:tasks.workunit.client.0.trial031.stdout: "average_primary_affinity_weighted": 1 2026-02-17T21:19:34.222 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.223 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.223 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.223 INFO:tasks.workunit.client.0.trial031.stdout: "osds": [ 2026-02-17T21:19:34.223 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.223 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:19:34.223 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "505cc7f4-fb42-4465-8933-5f18295caf89", 2026-02-17T21:19:34.223 INFO:tasks.workunit.client.0.trial031.stdout: "up": 1, 2026-02-17T21:19:34.223 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:19:34.223 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:19:34.223 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:19:34.223 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 5, 2026-02-17T21:19:34.223 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 28, 2026-02-17T21:19:34.224 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 35, 2026-02-17T21:19:34.224 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 42, 2026-02-17T21:19:34.224 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 29, 2026-02-17T21:19:34.224 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:19:34.224 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:19:34.224 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:19:34.224 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.224 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:19:34.224 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6802", 2026-02-17T21:19:34.224 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:19:34.224 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.224 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.225 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:19:34.225 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6803", 2026-02-17T21:19:34.225 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:19:34.225 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.225 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.225 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.225 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:19:34.225 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:19:34.225 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.225 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:19:34.225 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6804", 2026-02-17T21:19:34.225 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:19:34.225 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.226 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.226 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:19:34.226 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6805", 2026-02-17T21:19:34.226 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:19:34.226 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.226 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.226 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.226 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:19:34.226 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:19:34.226 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.226 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:19:34.226 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6808", 2026-02-17T21:19:34.226 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:19:34.226 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.227 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.227 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:19:34.227 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6809", 2026-02-17T21:19:34.227 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:19:34.227 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.227 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.227 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.227 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:19:34.227 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:19:34.227 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.227 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:19:34.227 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6806", 2026-02-17T21:19:34.227 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:19:34.228 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.228 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.228 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:19:34.228 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6807", 2026-02-17T21:19:34.228 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:19:34.228 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.228 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.228 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.228 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6803/1153784009", 2026-02-17T21:19:34.228 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6805/1153784009", 2026-02-17T21:19:34.228 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6809/1153784009", 2026-02-17T21:19:34.228 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6807/1153784009", 2026-02-17T21:19:34.228 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:19:34.229 INFO:tasks.workunit.client.0.trial031.stdout: "exists", 2026-02-17T21:19:34.229 INFO:tasks.workunit.client.0.trial031.stdout: "up" 2026-02-17T21:19:34.229 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.229 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.229 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.229 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:19:34.229 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "d8cc5e2c-83e0-49d6-8a94-49a26ec2d2cb", 2026-02-17T21:19:34.229 INFO:tasks.workunit.client.0.trial031.stdout: "up": 0, 2026-02-17T21:19:34.229 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:19:34.229 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:19:34.229 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:19:34.229 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 0, 2026-02-17T21:19:34.230 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 0, 2026-02-17T21:19:34.230 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 9, 2026-02-17T21:19:34.230 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 22, 2026-02-17T21:19:34.230 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 30, 2026-02-17T21:19:34.230 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:19:34.230 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:19:34.230 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:19:34.230 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.230 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:19:34.230 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6810", 2026-02-17T21:19:34.230 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:19:34.230 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.231 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.231 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:19:34.231 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6811", 2026-02-17T21:19:34.231 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:19:34.231 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.231 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.231 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.231 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:19:34.231 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:19:34.231 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.231 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:19:34.231 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6812", 2026-02-17T21:19:34.231 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:19:34.232 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.232 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.232 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:19:34.232 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6813", 2026-02-17T21:19:34.232 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:19:34.232 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.232 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.232 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.232 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:19:34.232 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:19:34.232 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.233 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:19:34.233 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6816", 2026-02-17T21:19:34.233 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:19:34.233 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.233 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.233 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:19:34.233 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6817", 2026-02-17T21:19:34.233 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:19:34.233 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.233 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.233 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.233 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:19:34.234 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:19:34.234 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.234 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:19:34.234 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6814", 2026-02-17T21:19:34.234 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:19:34.234 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.234 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.234 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:19:34.234 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6815", 2026-02-17T21:19:34.234 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:19:34.234 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.234 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.235 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.235 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6811/2990836993", 2026-02-17T21:19:34.235 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6813/2990836993", 2026-02-17T21:19:34.235 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2990836993", 2026-02-17T21:19:34.235 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2990836993", 2026-02-17T21:19:34.235 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:19:34.235 INFO:tasks.workunit.client.0.trial031.stdout: "exists" 2026-02-17T21:19:34.235 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.235 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.235 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.235 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:19:34.235 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "cd50e885-11ef-49a7-a859-6f1549a4e9de", 2026-02-17T21:19:34.236 INFO:tasks.workunit.client.0.trial031.stdout: "up": 1, 2026-02-17T21:19:34.236 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:19:34.236 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:19:34.236 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:19:34.236 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 14, 2026-02-17T21:19:34.236 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 28, 2026-02-17T21:19:34.236 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 38, 2026-02-17T21:19:34.236 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 0, 2026-02-17T21:19:34.236 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 31, 2026-02-17T21:19:34.236 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:19:34.236 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:19:34.236 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:19:34.236 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.237 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:19:34.237 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6810", 2026-02-17T21:19:34.237 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:19:34.237 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.237 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.237 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:19:34.237 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6811", 2026-02-17T21:19:34.237 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:19:34.237 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.237 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.237 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.237 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:19:34.237 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:19:34.238 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.238 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:19:34.238 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6812", 2026-02-17T21:19:34.238 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:19:34.238 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.238 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.238 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:19:34.238 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6813", 2026-02-17T21:19:34.238 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:19:34.238 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.238 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.238 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.238 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:19:34.238 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:19:34.239 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.239 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:19:34.239 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6816", 2026-02-17T21:19:34.239 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:19:34.239 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.239 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.239 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:19:34.239 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6817", 2026-02-17T21:19:34.239 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:19:34.239 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.239 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.239 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.240 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:19:34.240 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:19:34.240 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.240 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:19:34.240 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6814", 2026-02-17T21:19:34.240 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:19:34.240 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.240 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.240 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:19:34.240 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6815", 2026-02-17T21:19:34.240 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:19:34.240 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.240 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.241 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.241 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6811/2603940079", 2026-02-17T21:19:34.241 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6813/2603940079", 2026-02-17T21:19:34.241 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2603940079", 2026-02-17T21:19:34.241 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2603940079", 2026-02-17T21:19:34.241 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:19:34.241 INFO:tasks.workunit.client.0.trial031.stdout: "exists", 2026-02-17T21:19:34.241 INFO:tasks.workunit.client.0.trial031.stdout: "up" 2026-02-17T21:19:34.241 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.241 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.241 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.241 INFO:tasks.workunit.client.0.trial031.stdout: "osd_xinfo": [ 2026-02-17T21:19:34.241 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.242 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:19:34.242 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:18:51.522342+0000", 2026-02-17T21:19:34.242 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:19:34.242 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:19:34.242 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:19:34.242 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:19:34.242 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "2026-02-17T21:18:07.038217+0000", 2026-02-17T21:19:34.242 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:19:34.242 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.242 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.242 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:19:34.242 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:18:51.801948+0000", 2026-02-17T21:19:34.243 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:19:34.243 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:19:34.243 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:19:34.243 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:19:34.243 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "2026-02-17T21:18:16.051674+0000", 2026-02-17T21:19:34.243 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:19:34.243 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.243 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.243 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:19:34.243 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:18:52.105412+0000", 2026-02-17T21:19:34.243 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:19:34.243 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:19:34.243 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:19:34.243 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:19:34.244 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "2026-02-17T21:18:25.422918+0000", 2026-02-17T21:19:34.244 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:19:34.244 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.244 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.244 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap": [], 2026-02-17T21:19:34.244 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap_items": [], 2026-02-17T21:19:34.244 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap_primaries": [], 2026-02-17T21:19:34.244 INFO:tasks.workunit.client.0.trial031.stdout: "pg_temp": [], 2026-02-17T21:19:34.244 INFO:tasks.workunit.client.0.trial031.stdout: "primary_temp": [], 2026-02-17T21:19:34.244 INFO:tasks.workunit.client.0.trial031.stdout: "blocklist": {}, 2026-02-17T21:19:34.244 INFO:tasks.workunit.client.0.trial031.stdout: "range_blocklist": {}, 2026-02-17T21:19:34.244 INFO:tasks.workunit.client.0.trial031.stdout: "erasure_code_profiles": { 2026-02-17T21:19:34.244 INFO:tasks.workunit.client.0.trial031.stdout: "default": { 2026-02-17T21:19:34.244 INFO:tasks.workunit.client.0.trial031.stdout: "k": "2", 2026-02-17T21:19:34.244 INFO:tasks.workunit.client.0.trial031.stdout: "m": "2", 2026-02-17T21:19:34.245 INFO:tasks.workunit.client.0.trial031.stdout: "plugin": "jerasure", 2026-02-17T21:19:34.245 INFO:tasks.workunit.client.0.trial031.stdout: "technique": "reed_sol_van" 2026-02-17T21:19:34.245 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.245 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.245 INFO:tasks.workunit.client.0.trial031.stdout: "removed_snaps_queue": [], 2026-02-17T21:19:34.245 INFO:tasks.workunit.client.0.trial031.stdout: "new_removed_snaps": [], 2026-02-17T21:19:34.245 INFO:tasks.workunit.client.0.trial031.stdout: "new_purged_snaps": [], 2026-02-17T21:19:34.245 INFO:tasks.workunit.client.0.trial031.stdout: "crush_node_flags": {}, 2026-02-17T21:19:34.245 INFO:tasks.workunit.client.0.trial031.stdout: "device_class_flags": {}, 2026-02-17T21:19:34.245 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode": { 2026-02-17T21:19:34.245 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode_enabled": false, 2026-02-17T21:19:34.245 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_bucket_count": 0, 2026-02-17T21:19:34.245 INFO:tasks.workunit.client.0.trial031.stdout: "degraded_stretch_mode": 0, 2026-02-17T21:19:34.245 INFO:tasks.workunit.client.0.trial031.stdout: "recovering_stretch_mode": 0, 2026-02-17T21:19:34.245 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode_bucket": 0 2026-02-17T21:19:34.246 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.246 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.246 INFO:tasks.workunit.client.0.trial031.stdout: "osd_metadata": [ 2026-02-17T21:19:34.246 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.246 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:19:34.246 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:19:34.246 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6804/1153784009,v1:127.0.0.1:6805/1153784009]", 2026-02-17T21:19:34.246 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:19:34.246 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:19:34.246 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:19:34.246 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:19:34.246 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:19:34.246 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:19:34.246 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:19:34.247 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:18:00.442071Z", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6802/1153784009,v1:127.0.0.1:6803/1153784009]", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/1153784009,v1:127.0.0.1:6809/1153784009]", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/1153784009,v1:127.0.0.1:6807/1153784009]", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:19:34.248 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/0", 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6812/2990836993,v1:127.0.0.1:6813/2990836993]", 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:19:34.249 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:19:34.250 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:18:09.633013Z", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6810/2990836993,v1:127.0.0.1:6811/2990836993]", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2990836993,v1:127.0.0.1:6817/2990836993]", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2990836993,v1:127.0.0.1:6815/2990836993]", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:19:34.251 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/1", 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6812/2603940079,v1:127.0.0.1:6813/2603940079]", 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:19:34.252 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:19:34.253 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:18:18.880812Z", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6810/2603940079,v1:127.0.0.1:6811/2603940079]", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2603940079,v1:127.0.0.1:6817/2603940079]", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2603940079,v1:127.0.0.1:6815/2603940079]", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:19:34.254 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/2", 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_clean_epochs": { 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: "min_last_epoch_clean": 0, 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_clean": { 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: "per_pool": [] 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: "osd_epochs": [ 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 35 2026-02-17T21:19:34.255 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 40 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_first_committed": 1, 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_last_committed": 43, 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: "crushmap": { 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: "devices": [ 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.0", 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.1", 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.256 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.2", 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: "types": [ 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 0, 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd" 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: "name": "host" 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 2, 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: "name": "chassis" 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 3, 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: "name": "rack" 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.257 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 4, 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: "name": "row" 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 5, 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: "name": "pdu" 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 6, 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: "name": "pod" 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 7, 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: "name": "room" 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 8, 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: "name": "datacenter" 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 9, 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: "name": "zone" 2026-02-17T21:19:34.258 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 10, 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "name": "region" 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "name": "root" 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "buckets": [ 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "id": -1, 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "name": "default", 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "root", 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "id": -3, 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:19:34.259 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: "id": -2, 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: "name": "default~ssd", 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "root", 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: "id": -4, 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: "id": -3, 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: "name": "trial031", 2026-02-17T21:19:34.260 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "host", 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 1 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 2 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.261 INFO:tasks.workunit.client.0.trial031.stdout: "id": -4, 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "name": "trial031~ssd", 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "host", 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 1 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 2 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.262 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: "rules": [ 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: "rule_id": 0, 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: "rule_name": "replicated_rule", 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: "type": 1, 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: "steps": [ 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: "op": "take", 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: "item": -1, 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: "item_name": "default" 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: "op": "choose_firstn", 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: "num": 0, 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: "type": "osd" 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: "op": "emit" 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.263 INFO:tasks.workunit.client.0.trial031.stdout: "tunables": { 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "choose_local_tries": 0, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "choose_local_fallback_tries": 0, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "choose_total_tries": 50, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_descend_once": 1, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_vary_r": 1, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_stable": 1, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "msr_descents": 100, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "msr_collision_tries": 100, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "straw_calc_version": 1, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "allowed_bucket_algs": 54, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "profile": "jewel", 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "optimal_tunables": 1, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "legacy_tunables": 0, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "minimum_required_version": "jewel", 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables": 1, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables2": 1, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "has_v2_rules": 0, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables3": 1, 2026-02-17T21:19:34.264 INFO:tasks.workunit.client.0.trial031.stdout: "has_v3_rules": 0, 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stderr:report 2485823527 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: "has_v4_buckets": 1, 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables5": 1, 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: "has_v5_rules": 0, 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: "has_msr_rules": 0 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: "choose_args": {} 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: "fsmap": { 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 1, 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: "btime": "2026-02-17T21:17:57:753188+0000", 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: "default_fscid": -1, 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: "compat": { 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: "compat": {}, 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: "ro_compat": {}, 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: "incompat": { 2026-02-17T21:19:34.265 INFO:tasks.workunit.client.0.trial031.stdout: "feature_1": "base v0.20", 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "feature_2": "client writeable ranges", 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "feature_3": "default file layouts on dirs", 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "feature_4": "dir inode in separate object", 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "feature_5": "mds uses versioned encoding", 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "feature_8": "no anchor table", 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "feature_9": "file layout v2", 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "feature_10": "snaprealm v2", 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "feature_11": "minor log segments", 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "feature_12": "quiesce subvolumes" 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "feature_flags": { 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "enable_multiple": true, 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "ever_enabled_multiple": true 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "standbys": [], 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "filesystems": [] 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "mdsmap_first_committed": 1, 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "mdsmap_last_committed": 1, 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "auth": { 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "first_committed": 1, 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "last_committed": 5, 2026-02-17T21:19:34.266 INFO:tasks.workunit.client.0.trial031.stdout: "num_secrets": 3 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg": 1, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_active": 1, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_unknown": 0, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_osd": 3, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "pool_sum": { 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "stat_sum": { 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes": 70600, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects": 100, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_clones": 0, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_copies": 300, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing": 0, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_degraded": 100, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_misplaced": 0, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_unfound": 0, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_dirty": 100, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_whiteouts": 0, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_read": 0, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_read_kb": 0, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_write": 100, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_write_kb": 100, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_scrub_errors": 0, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_deep_scrub_errors": 0, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_recovered": 0, 2026-02-17T21:19:34.267 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_recovered": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_keys_recovered": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_omap": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_kb": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_kb": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_promote": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_high": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_low": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_some": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_full": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_pinned": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_legacy_snapsets": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_large_omap_objects": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_manifest": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_bytes": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_keys": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_repaired": 0 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "store_stats": { 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "total": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "available": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 0, 2026-02-17T21:19:34.268 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 0, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 0 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "log_size": 16, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "ondisk_log_size": 16, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "up": 2, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "acting": 2, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "num_store_stats": 0 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "osd_sum": { 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 0, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 0, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "num_pgs": 3, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "num_osds": 3, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_osds": 3, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "kb": 314572800, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used": 83956, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_data": 2128, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_omap": 10, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_meta": 81717, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "kb_avail": 314488844, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "statfs": { 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "total": 322122547200, 2026-02-17T21:19:34.269 INFO:tasks.workunit.client.0.trial031.stdout: "available": 322036576256, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 2179072, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 497636, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 10389, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 83679083 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "hb_peers": [], 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "snap_trim_queue_len": 0, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "num_snap_trimming": 0, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "num_shards_repaired": 0, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "op_queue_age_hist": { 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "histogram": [], 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "upper_bound": 1 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "perf_stat": { 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ms": 0, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ms": 0, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ns": 0, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ns": 0 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "alerts": [], 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "network_ping_times": [] 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "osd_sum_by_class": { 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "ssd": { 2026-02-17T21:19:34.270 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 0, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 0, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "num_pgs": 3, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "num_osds": 3, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_osds": 3, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "kb": 314572800, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used": 83956, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_data": 2128, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_omap": 10, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_meta": 81717, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "kb_avail": 314488844, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "statfs": { 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "total": 322122547200, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "available": 322036576256, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 2179072, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 497636, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 10389, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 83679083 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "hb_peers": [], 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "snap_trim_queue_len": 0, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "num_snap_trimming": 0, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "num_shards_repaired": 0, 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "op_queue_age_hist": { 2026-02-17T21:19:34.271 INFO:tasks.workunit.client.0.trial031.stdout: "histogram": [], 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "upper_bound": 1 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "perf_stat": { 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ms": 0, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ms": 0, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ns": 0, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ns": 0 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "alerts": [], 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "network_ping_times": [] 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "pool_stats": [ 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "poolid": 1, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg": 1, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "stat_sum": { 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes": 70600, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects": 100, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_clones": 0, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_copies": 300, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing": 0, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_degraded": 100, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_misplaced": 0, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_unfound": 0, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_dirty": 100, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "num_whiteouts": 0, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "num_read": 0, 2026-02-17T21:19:34.272 INFO:tasks.workunit.client.0.trial031.stdout: "num_read_kb": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_write": 100, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_write_kb": 100, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_scrub_errors": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_deep_scrub_errors": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_recovered": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_recovered": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_keys_recovered": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_omap": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_kb": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_kb": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_promote": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_high": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_low": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_some": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_full": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_pinned": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_legacy_snapsets": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_large_omap_objects": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_manifest": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_bytes": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_keys": 0, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_repaired": 0 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "store_stats": { 2026-02-17T21:19:34.273 INFO:tasks.workunit.client.0.trial031.stdout: "total": 0, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "available": 0, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 1220608, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 210910, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 0, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 0 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "log_size": 16, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "ondisk_log_size": 16, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "up": 2, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "acting": 2, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "num_store_stats": 3 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "osd_stats": [ 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 150323855369 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 38654705677 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 163208757256 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.274 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_by_state": [ 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "state": "active+undersized+degraded", 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "num": 1 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_by_osd": [ 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 1, 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 1, 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 1, 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 0, 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snaps": [ 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "pool": 1, 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snaps": {} 2026-02-17T21:19:34.275 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "servicemap": { 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 4, 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:19:18.197201+0000", 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "services": { 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "osd": { 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "daemons": { 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "summary": "", 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "0": { 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "start_epoch": 0, 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "start_stamp": "0.000000", 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "gid": 0, 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "metadata": {}, 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "task_status": {} 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "2": { 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "start_epoch": 0, 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "start_stamp": "0.000000", 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "gid": 0, 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "metadata": {}, 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "task_status": {} 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "mgrstat_first_committed": 1, 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "mgrstat_last_committed": 45, 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "logm_first_committed": 1, 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "logm_last_committed": 233, 2026-02-17T21:19:34.276 INFO:tasks.workunit.client.0.trial031.stdout: "paxos": { 2026-02-17T21:19:34.277 INFO:tasks.workunit.client.0.trial031.stdout: "first_committed": 1, 2026-02-17T21:19:34.277 INFO:tasks.workunit.client.0.trial031.stdout: "last_committed": 332, 2026-02-17T21:19:34.277 INFO:tasks.workunit.client.0.trial031.stdout: "last_pn": 0, 2026-02-17T21:19:34.277 INFO:tasks.workunit.client.0.trial031.stdout: "accepted_pn": 0 2026-02-17T21:19:34.277 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:19:34.277 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:19:34.277 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-17T21:19:34.277 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:528: TEST_divergent_2: expr 2 + 2 2026-02-17T21:19:34.277 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:528: TEST_divergent_2: objname=existing_4 2026-02-17T21:19:34.277 INFO:tasks.workunit.client.0.trial031.stdout:writing non-divergent object existing_4 2026-02-17T21:19:34.277 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:529: TEST_divergent_2: echo 'writing non-divergent object existing_4' 2026-02-17T21:19:34.277 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:530: TEST_divergent_2: ceph pg dump pgs 2026-02-17T21:19:34.411 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:19:34.411 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70600 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:19:19.818123+0000 28'100 43:135 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:18:27.226024+0000 0'0 2026-02-17T21:18:27.226024+0000 0 0 periodic scrub scheduled @ 2026-02-19T05:02:25.367558+0000 0 0 2026-02-17T21:19:34.411 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:19:34.411 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:19:34.411 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:19:34.419 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:531: TEST_divergent_2: rados -p test put existing_4 /etc/resolv.conf 2026-02-17T21:19:34.448 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:533: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-17T21:19:34.448 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:533: TEST_divergent_2: wait_for_clean 2026-02-17T21:19:34.448 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:19:34.448 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:19:34.448 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:19:34.449 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-17T21:19:34.449 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:19:34.449 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:19:34.450 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:19:34.450 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:19:34.450 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:19:34.520 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-17T21:19:34.520 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:19:34.520 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:19:34.520 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:19:34.520 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:19:34.520 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:19:34.732 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:19:34.732 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:19:34.733 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:19:34.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:19:34.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:19:34.733 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:19:34.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855372 2026-02-17T21:19:34.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855372 2026-02-17T21:19:34.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855372' 2026-02-17T21:19:34.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:19:34.812 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:19:34.873 INFO:tasks.workunit.client.0.trial031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-17T21:19:34.876 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-17T21:19:34.876 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-17T21:19:34.876 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-17T21:19:34.876 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:19:34.877 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:19:34.956 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757259 2026-02-17T21:19:34.956 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757259 2026-02-17T21:19:34.956 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855372 2-163208757259' 2026-02-17T21:19:34.956 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:19:34.957 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855372 2026-02-17T21:19:34.957 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:19:34.958 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:19:34.958 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855372 2026-02-17T21:19:34.958 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:19:34.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855372 2026-02-17T21:19:34.960 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 150323855372 2026-02-17T21:19:34.960 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855372' 2026-02-17T21:19:34.960 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:19:35.169 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855370 -lt 150323855372 2026-02-17T21:19:35.169 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:19:36.170 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:19:36.170 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:19:36.376 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855372 -lt 150323855372 2026-02-17T21:19:36.376 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:19:36.377 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-163208757259 2026-02-17T21:19:36.377 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:19:36.378 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:19:36.379 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-163208757259 2026-02-17T21:19:36.379 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:19:36.381 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757259 2026-02-17T21:19:36.381 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 163208757259' 2026-02-17T21:19:36.381 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 163208757259 2026-02-17T21:19:36.381 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:19:36.594 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757259 -lt 163208757259 2026-02-17T21:19:36.595 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:19:36.595 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:36.595 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:36.863 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T21:19:36.863 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:36.863 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:36.863 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:36.863 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:36.863 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:36.863 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:36.863 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:37.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:37.064 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:37.064 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:37.064 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:37.329 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:37.329 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T21:19:37.329 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T21:19:37.329 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T21:19:37.329 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:37.329 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T21:19:37.429 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:19:37.429 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:37.430 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:37.431 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:37.431 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:37.431 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:37.431 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:37.431 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:37.631 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:37.631 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:37.632 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:37.632 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:37.895 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:37.895 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:19:37.895 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:19:37.895 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:19:37.896 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:19:37.896 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:19:37.896 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:19:37.896 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:19:37.896 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:19:38.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:19:38.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:19:38.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-17T21:19:38.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:38.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T21:19:38.366 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:19:38.366 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:38.367 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:38.367 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:38.367 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:38.367 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:38.367 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:38.367 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:38.569 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:38.570 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:38.570 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:38.570 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:38.841 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:38.841 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:19:38.841 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:19:38.841 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:19:38.841 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:19:38.842 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:19:38.842 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:19:38.842 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:19:38.842 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:19:39.109 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:19:39.109 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:19:39.109 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-17T21:19:39.109 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:39.109 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T21:19:39.510 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:19:39.510 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:39.510 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:39.510 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:39.510 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:39.510 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:39.511 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:39.511 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:39.716 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:39.717 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:39.717 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:39.717 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:39.983 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:39.983 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:19:39.983 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:19:39.983 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:19:39.983 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:19:39.983 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:19:39.983 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:19:39.983 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:19:39.983 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:19:40.254 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:19:40.254 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:19:40.254 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-17T21:19:40.255 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:40.255 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T21:19:41.055 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:19:41.055 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:41.056 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:41.056 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:41.056 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:41.056 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:41.056 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:41.056 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:41.254 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:41.254 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:41.255 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:41.255 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:41.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:41.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:19:41.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:19:41.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:19:41.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:19:41.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:19:41.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:19:41.515 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:19:41.515 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:19:41.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:19:41.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:19:41.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-17T21:19:41.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:41.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-17T21:19:43.374 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:19:43.374 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:43.374 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:43.374 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:43.375 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:43.375 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:43.375 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:43.375 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:43.569 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:43.570 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:43.570 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:43.570 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:43.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:43.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:19:43.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:19:43.832 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:19:43.832 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:19:43.832 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:19:43.832 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:19:43.832 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:19:43.832 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:19:44.091 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:19:44.091 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:19:44.091 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-17T21:19:44.092 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:44.092 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-17T21:19:47.292 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:19:47.292 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:47.293 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:47.293 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:47.293 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:47.293 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:47.294 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:47.294 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:47.491 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:47.492 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:47.492 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:47.492 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:47.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:47.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:19:47.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:19:47.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:19:47.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:19:47.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:19:47.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:19:47.750 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:19:47.751 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:19:48.023 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:19:48.023 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:19:48.024 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-17T21:19:48.024 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:48.024 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-17T21:19:54.424 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:19:54.425 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:19:54.426 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:19:54.426 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:19:54.426 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:19:54.426 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:19:54.426 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:19:54.426 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:19:54.627 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:19:54.627 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:19:54.628 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:19:54.628 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:19:54.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:19:54.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:19:54.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:19:54.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:19:54.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:19:54.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:19:54.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:19:54.885 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:19:54.885 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:19:55.142 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:19:55.142 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:19:55.142 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-17T21:19:55.142 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:19:55.142 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-17T21:20:02.442 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:20:02.443 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:20:02.444 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:20:02.444 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:20:02.444 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:20:02.444 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:20:02.444 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:20:02.444 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:20:02.640 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:20:02.641 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:20:02.641 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:20:02.641 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:20:02.902 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:20:02.902 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:20:02.902 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:20:02.903 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:20:02.903 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:20:02.903 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:20:02.903 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:20:02.903 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:20:02.903 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:20:03.160 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:20:03.160 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:20:03.160 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-17T21:20:03.160 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-17T21:20:03.415 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:20:03.415 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_fingerprint": "06112e0a-c553-46df-b7de-3091abea00ce", 2026-02-17T21:20:03.415 INFO:tasks.workunit.client.0.trial031.stdout: "version": "19.2.3-679-gd855f53b", 2026-02-17T21:20:03.416 INFO:tasks.workunit.client.0.trial031.stdout: "commit": "d855f53b89fdcec760fd9232a5fb55ed4fb111a1", 2026-02-17T21:20:03.416 INFO:tasks.workunit.client.0.trial031.stdout: "timestamp": "2026-02-17T21:20:03.415630+0000", 2026-02-17T21:20:03.416 INFO:tasks.workunit.client.0.trial031.stdout: "tag": "", 2026-02-17T21:20:03.416 INFO:tasks.workunit.client.0.trial031.stdout: "health": { 2026-02-17T21:20:03.416 INFO:tasks.workunit.client.0.trial031.stdout: "status": "HEALTH_WARN", 2026-02-17T21:20:03.416 INFO:tasks.workunit.client.0.trial031.stdout: "checks": { 2026-02-17T21:20:03.416 INFO:tasks.workunit.client.0.trial031.stdout: "OSDMAP_FLAGS": { 2026-02-17T21:20:03.416 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:20:03.416 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:20:03.416 INFO:tasks.workunit.client.0.trial031.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-17T21:20:03.416 INFO:tasks.workunit.client.0.trial031.stdout: "count": 17 2026-02-17T21:20:03.417 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.417 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [], 2026-02-17T21:20:03.417 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:20:03.417 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.417 INFO:tasks.workunit.client.0.trial031.stdout: "OSD_DOWN": { 2026-02-17T21:20:03.417 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:20:03.417 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:20:03.417 INFO:tasks.workunit.client.0.trial031.stdout: "message": "1 osds down", 2026-02-17T21:20:03.417 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:20:03.417 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.417 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:20:03.417 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.418 INFO:tasks.workunit.client.0.trial031.stdout: "message": "osd.1 (root=default,host=trial031) is down" 2026-02-17T21:20:03.418 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.418 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.418 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:20:03.418 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.418 INFO:tasks.workunit.client.0.trial031.stdout: "PG_DEGRADED": { 2026-02-17T21:20:03.418 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:20:03.418 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:20:03.418 INFO:tasks.workunit.client.0.trial031.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2026-02-17T21:20:03.418 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:20:03.418 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.418 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:20:03.419 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.419 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:20:03.419 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.419 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.419 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:20:03.419 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.419 INFO:tasks.workunit.client.0.trial031.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-17T21:20:03.419 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:20:03.419 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:20:03.419 INFO:tasks.workunit.client.0.trial031.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-17T21:20:03.419 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:20:03.419 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.419 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:20:03.420 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.420 INFO:tasks.workunit.client.0.trial031.stdout: "message": "application not enabled on pool 'test'" 2026-02-17T21:20:03.420 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.420 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.420 INFO:tasks.workunit.client.0.trial031.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-17T21:20:03.420 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.420 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.420 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:20:03.420 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.420 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.420 INFO:tasks.workunit.client.0.trial031.stdout: "mutes": [] 2026-02-17T21:20:03.420 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.421 INFO:tasks.workunit.client.0.trial031.stdout: "monmap_first_committed": 1, 2026-02-17T21:20:03.421 INFO:tasks.workunit.client.0.trial031.stdout: "monmap_last_committed": 1, 2026-02-17T21:20:03.421 INFO:tasks.workunit.client.0.trial031.stdout: "monmap": { 2026-02-17T21:20:03.421 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 1, 2026-02-17T21:20:03.421 INFO:tasks.workunit.client.0.trial031.stdout: "fsid": "5ebac6e7-5321-4893-ad68-c06127ed455b", 2026-02-17T21:20:03.421 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:17:57.669569Z", 2026-02-17T21:20:03.421 INFO:tasks.workunit.client.0.trial031.stdout: "created": "2026-02-17T21:17:57.669569Z", 2026-02-17T21:20:03.421 INFO:tasks.workunit.client.0.trial031.stdout: "min_mon_release": 19, 2026-02-17T21:20:03.421 INFO:tasks.workunit.client.0.trial031.stdout: "min_mon_release_name": "squid", 2026-02-17T21:20:03.421 INFO:tasks.workunit.client.0.trial031.stdout: "election_strategy": 1, 2026-02-17T21:20:03.421 INFO:tasks.workunit.client.0.trial031.stdout: "disallowed_leaders": "", 2026-02-17T21:20:03.421 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode": false, 2026-02-17T21:20:03.421 INFO:tasks.workunit.client.0.trial031.stdout: "tiebreaker_mon": "", 2026-02-17T21:20:03.421 INFO:tasks.workunit.client.0.trial031.stdout: "removed_ranks": "", 2026-02-17T21:20:03.422 INFO:tasks.workunit.client.0.trial031.stdout: "features": { 2026-02-17T21:20:03.422 INFO:tasks.workunit.client.0.trial031.stdout: "persistent": [ 2026-02-17T21:20:03.422 INFO:tasks.workunit.client.0.trial031.stdout: "kraken", 2026-02-17T21:20:03.422 INFO:tasks.workunit.client.0.trial031.stdout: "luminous", 2026-02-17T21:20:03.422 INFO:tasks.workunit.client.0.trial031.stdout: "mimic", 2026-02-17T21:20:03.422 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap-prune", 2026-02-17T21:20:03.422 INFO:tasks.workunit.client.0.trial031.stdout: "nautilus", 2026-02-17T21:20:03.422 INFO:tasks.workunit.client.0.trial031.stdout: "octopus", 2026-02-17T21:20:03.422 INFO:tasks.workunit.client.0.trial031.stdout: "pacific", 2026-02-17T21:20:03.422 INFO:tasks.workunit.client.0.trial031.stdout: "elector-pinging", 2026-02-17T21:20:03.422 INFO:tasks.workunit.client.0.trial031.stdout: "quincy", 2026-02-17T21:20:03.422 INFO:tasks.workunit.client.0.trial031.stdout: "reef", 2026-02-17T21:20:03.422 INFO:tasks.workunit.client.0.trial031.stdout: "squid" 2026-02-17T21:20:03.423 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.423 INFO:tasks.workunit.client.0.trial031.stdout: "optional": [] 2026-02-17T21:20:03.423 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.423 INFO:tasks.workunit.client.0.trial031.stdout: "mons": [ 2026-02-17T21:20:03.423 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.423 INFO:tasks.workunit.client.0.trial031.stdout: "rank": 0, 2026-02-17T21:20:03.423 INFO:tasks.workunit.client.0.trial031.stdout: "name": "a", 2026-02-17T21:20:03.423 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:20:03.423 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:20:03.423 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.423 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:20:03.423 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:7115", 2026-02-17T21:20:03.424 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 0 2026-02-17T21:20:03.424 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.424 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.424 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.424 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:7115/0", 2026-02-17T21:20:03.424 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-17T21:20:03.424 INFO:tasks.workunit.client.0.trial031.stdout: "priority": 0, 2026-02-17T21:20:03.424 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 0, 2026-02-17T21:20:03.424 INFO:tasks.workunit.client.0.trial031.stdout: "crush_location": "{}" 2026-02-17T21:20:03.424 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.424 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.424 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.424 INFO:tasks.workunit.client.0.trial031.stdout: "quorum": [ 2026-02-17T21:20:03.424 INFO:tasks.workunit.client.0.trial031.stdout: 0 2026-02-17T21:20:03.425 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.425 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap": { 2026-02-17T21:20:03.425 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 43, 2026-02-17T21:20:03.425 INFO:tasks.workunit.client.0.trial031.stdout: "fsid": "5ebac6e7-5321-4893-ad68-c06127ed455b", 2026-02-17T21:20:03.425 INFO:tasks.workunit.client.0.trial031.stdout: "created": "2026-02-17T21:17:57.753415+0000", 2026-02-17T21:20:03.425 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:19:04.016405+0000", 2026-02-17T21:20:03.425 INFO:tasks.workunit.client.0.trial031.stdout: "last_up_change": "2026-02-17T21:19:02.902098+0000", 2026-02-17T21:20:03.425 INFO:tasks.workunit.client.0.trial031.stdout: "last_in_change": "2026-02-17T21:19:03.915798+0000", 2026-02-17T21:20:03.425 INFO:tasks.workunit.client.0.trial031.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-17T21:20:03.425 INFO:tasks.workunit.client.0.trial031.stdout: "flags_num": 5800384, 2026-02-17T21:20:03.425 INFO:tasks.workunit.client.0.trial031.stdout: "flags_set": [ 2026-02-17T21:20:03.425 INFO:tasks.workunit.client.0.trial031.stdout: "nodown", 2026-02-17T21:20:03.425 INFO:tasks.workunit.client.0.trial031.stdout: "noin", 2026-02-17T21:20:03.426 INFO:tasks.workunit.client.0.trial031.stdout: "noout", 2026-02-17T21:20:03.426 INFO:tasks.workunit.client.0.trial031.stdout: "pglog_hardlimit", 2026-02-17T21:20:03.426 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snapdirs", 2026-02-17T21:20:03.426 INFO:tasks.workunit.client.0.trial031.stdout: "recovery_deletes", 2026-02-17T21:20:03.426 INFO:tasks.workunit.client.0.trial031.stdout: "sortbitwise" 2026-02-17T21:20:03.426 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.426 INFO:tasks.workunit.client.0.trial031.stdout: "crush_version": 9, 2026-02-17T21:20:03.426 INFO:tasks.workunit.client.0.trial031.stdout: "full_ratio": 0.99000000953674316, 2026-02-17T21:20:03.426 INFO:tasks.workunit.client.0.trial031.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-17T21:20:03.426 INFO:tasks.workunit.client.0.trial031.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-17T21:20:03.426 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_snapshot": "", 2026-02-17T21:20:03.426 INFO:tasks.workunit.client.0.trial031.stdout: "pool_max": 1, 2026-02-17T21:20:03.426 INFO:tasks.workunit.client.0.trial031.stdout: "max_osd": 3, 2026-02-17T21:20:03.427 INFO:tasks.workunit.client.0.trial031.stdout: "require_min_compat_client": "luminous", 2026-02-17T21:20:03.427 INFO:tasks.workunit.client.0.trial031.stdout: "min_compat_client": "jewel", 2026-02-17T21:20:03.427 INFO:tasks.workunit.client.0.trial031.stdout: "require_osd_release": "squid", 2026-02-17T21:20:03.427 INFO:tasks.workunit.client.0.trial031.stdout: "allow_crimson": false, 2026-02-17T21:20:03.427 INFO:tasks.workunit.client.0.trial031.stdout: "pools": [ 2026-02-17T21:20:03.427 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.427 INFO:tasks.workunit.client.0.trial031.stdout: "pool": 1, 2026-02-17T21:20:03.427 INFO:tasks.workunit.client.0.trial031.stdout: "pool_name": "test", 2026-02-17T21:20:03.427 INFO:tasks.workunit.client.0.trial031.stdout: "create_time": "2026-02-17T21:18:27.190988+0000", 2026-02-17T21:20:03.427 INFO:tasks.workunit.client.0.trial031.stdout: "flags": 1, 2026-02-17T21:20:03.427 INFO:tasks.workunit.client.0.trial031.stdout: "flags_names": "hashpspool", 2026-02-17T21:20:03.427 INFO:tasks.workunit.client.0.trial031.stdout: "type": 1, 2026-02-17T21:20:03.427 INFO:tasks.workunit.client.0.trial031.stdout: "size": 3, 2026-02-17T21:20:03.428 INFO:tasks.workunit.client.0.trial031.stdout: "min_size": 2, 2026-02-17T21:20:03.428 INFO:tasks.workunit.client.0.trial031.stdout: "crush_rule": 0, 2026-02-17T21:20:03.428 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_count": 0, 2026-02-17T21:20:03.428 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_target": 0, 2026-02-17T21:20:03.428 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_barrier": 0, 2026-02-17T21:20:03.428 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-17T21:20:03.428 INFO:tasks.workunit.client.0.trial031.stdout: "is_stretch_pool": false, 2026-02-17T21:20:03.428 INFO:tasks.workunit.client.0.trial031.stdout: "object_hash": 2, 2026-02-17T21:20:03.428 INFO:tasks.workunit.client.0.trial031.stdout: "pg_autoscale_mode": "off", 2026-02-17T21:20:03.428 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num": 1, 2026-02-17T21:20:03.428 INFO:tasks.workunit.client.0.trial031.stdout: "pg_placement_num": 1, 2026-02-17T21:20:03.428 INFO:tasks.workunit.client.0.trial031.stdout: "pg_placement_num_target": 1, 2026-02-17T21:20:03.428 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num_target": 1, 2026-02-17T21:20:03.429 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num_pending": 1, 2026-02-17T21:20:03.429 INFO:tasks.workunit.client.0.trial031.stdout: "last_pg_merge_meta": { 2026-02-17T21:20:03.429 INFO:tasks.workunit.client.0.trial031.stdout: "source_pgid": "0.0", 2026-02-17T21:20:03.429 INFO:tasks.workunit.client.0.trial031.stdout: "ready_epoch": 0, 2026-02-17T21:20:03.429 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_started": 0, 2026-02-17T21:20:03.429 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_clean": 0, 2026-02-17T21:20:03.429 INFO:tasks.workunit.client.0.trial031.stdout: "source_version": "0'0", 2026-02-17T21:20:03.429 INFO:tasks.workunit.client.0.trial031.stdout: "target_version": "0'0" 2026-02-17T21:20:03.429 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.429 INFO:tasks.workunit.client.0.trial031.stdout: "last_change": "28", 2026-02-17T21:20:03.429 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend": "0", 2026-02-17T21:20:03.429 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-17T21:20:03.429 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend_preluminous": "0", 2026-02-17T21:20:03.429 INFO:tasks.workunit.client.0.trial031.stdout: "auid": 0, 2026-02-17T21:20:03.430 INFO:tasks.workunit.client.0.trial031.stdout: "snap_mode": "selfmanaged", 2026-02-17T21:20:03.430 INFO:tasks.workunit.client.0.trial031.stdout: "snap_seq": 0, 2026-02-17T21:20:03.430 INFO:tasks.workunit.client.0.trial031.stdout: "snap_epoch": 0, 2026-02-17T21:20:03.430 INFO:tasks.workunit.client.0.trial031.stdout: "pool_snaps": [], 2026-02-17T21:20:03.430 INFO:tasks.workunit.client.0.trial031.stdout: "removed_snaps": "[]", 2026-02-17T21:20:03.430 INFO:tasks.workunit.client.0.trial031.stdout: "quota_max_bytes": 0, 2026-02-17T21:20:03.430 INFO:tasks.workunit.client.0.trial031.stdout: "quota_max_objects": 0, 2026-02-17T21:20:03.430 INFO:tasks.workunit.client.0.trial031.stdout: "tiers": [], 2026-02-17T21:20:03.430 INFO:tasks.workunit.client.0.trial031.stdout: "tier_of": -1, 2026-02-17T21:20:03.430 INFO:tasks.workunit.client.0.trial031.stdout: "read_tier": -1, 2026-02-17T21:20:03.430 INFO:tasks.workunit.client.0.trial031.stdout: "write_tier": -1, 2026-02-17T21:20:03.430 INFO:tasks.workunit.client.0.trial031.stdout: "cache_mode": "none", 2026-02-17T21:20:03.430 INFO:tasks.workunit.client.0.trial031.stdout: "target_max_bytes": 0, 2026-02-17T21:20:03.431 INFO:tasks.workunit.client.0.trial031.stdout: "target_max_objects": 0, 2026-02-17T21:20:03.431 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-17T21:20:03.431 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-17T21:20:03.431 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-17T21:20:03.431 INFO:tasks.workunit.client.0.trial031.stdout: "cache_min_flush_age": 0, 2026-02-17T21:20:03.431 INFO:tasks.workunit.client.0.trial031.stdout: "cache_min_evict_age": 0, 2026-02-17T21:20:03.431 INFO:tasks.workunit.client.0.trial031.stdout: "erasure_code_profile": "", 2026-02-17T21:20:03.431 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_params": { 2026-02-17T21:20:03.431 INFO:tasks.workunit.client.0.trial031.stdout: "type": "none" 2026-02-17T21:20:03.431 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.431 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_period": 0, 2026-02-17T21:20:03.431 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_count": 0, 2026-02-17T21:20:03.431 INFO:tasks.workunit.client.0.trial031.stdout: "use_gmt_hitset": true, 2026-02-17T21:20:03.432 INFO:tasks.workunit.client.0.trial031.stdout: "min_read_recency_for_promote": 0, 2026-02-17T21:20:03.432 INFO:tasks.workunit.client.0.trial031.stdout: "min_write_recency_for_promote": 0, 2026-02-17T21:20:03.432 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_grade_decay_rate": 0, 2026-02-17T21:20:03.432 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_search_last_n": 0, 2026-02-17T21:20:03.432 INFO:tasks.workunit.client.0.trial031.stdout: "grade_table": [], 2026-02-17T21:20:03.432 INFO:tasks.workunit.client.0.trial031.stdout: "stripe_width": 0, 2026-02-17T21:20:03.432 INFO:tasks.workunit.client.0.trial031.stdout: "expected_num_objects": 0, 2026-02-17T21:20:03.432 INFO:tasks.workunit.client.0.trial031.stdout: "fast_read": false, 2026-02-17T21:20:03.432 INFO:tasks.workunit.client.0.trial031.stdout: "options": {}, 2026-02-17T21:20:03.432 INFO:tasks.workunit.client.0.trial031.stdout: "application_metadata": {}, 2026-02-17T21:20:03.432 INFO:tasks.workunit.client.0.trial031.stdout: "read_balance": { 2026-02-17T21:20:03.432 INFO:tasks.workunit.client.0.trial031.stdout: "score_type": "Fair distribution", 2026-02-17T21:20:03.432 INFO:tasks.workunit.client.0.trial031.stdout: "score_acting": 2.9900000095367432, 2026-02-17T21:20:03.433 INFO:tasks.workunit.client.0.trial031.stdout: "score_stable": 2.9900000095367432, 2026-02-17T21:20:03.433 INFO:tasks.workunit.client.0.trial031.stdout: "optimal_score": 0.67000001668930054, 2026-02-17T21:20:03.433 INFO:tasks.workunit.client.0.trial031.stdout: "raw_score_acting": 2, 2026-02-17T21:20:03.433 INFO:tasks.workunit.client.0.trial031.stdout: "raw_score_stable": 2, 2026-02-17T21:20:03.433 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity_weighted": 1, 2026-02-17T21:20:03.433 INFO:tasks.workunit.client.0.trial031.stdout: "average_primary_affinity": 1, 2026-02-17T21:20:03.433 INFO:tasks.workunit.client.0.trial031.stdout: "average_primary_affinity_weighted": 1 2026-02-17T21:20:03.433 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.433 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.433 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.433 INFO:tasks.workunit.client.0.trial031.stdout: "osds": [ 2026-02-17T21:20:03.433 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.433 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:20:03.434 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "505cc7f4-fb42-4465-8933-5f18295caf89", 2026-02-17T21:20:03.434 INFO:tasks.workunit.client.0.trial031.stdout: "up": 1, 2026-02-17T21:20:03.434 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:20:03.434 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:20:03.434 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:20:03.434 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 5, 2026-02-17T21:20:03.434 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 28, 2026-02-17T21:20:03.434 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 35, 2026-02-17T21:20:03.434 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 42, 2026-02-17T21:20:03.434 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 29, 2026-02-17T21:20:03.434 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:20:03.434 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:20:03.434 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:20:03.435 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.435 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:20:03.435 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6802", 2026-02-17T21:20:03.435 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:20:03.435 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.435 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.435 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:20:03.435 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6803", 2026-02-17T21:20:03.435 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:20:03.435 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.435 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.435 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.436 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:20:03.436 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:20:03.436 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.436 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:20:03.436 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6804", 2026-02-17T21:20:03.436 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:20:03.436 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.436 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.436 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:20:03.436 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6805", 2026-02-17T21:20:03.436 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:20:03.436 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.436 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.437 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.437 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:20:03.437 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:20:03.437 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.437 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:20:03.437 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6808", 2026-02-17T21:20:03.437 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:20:03.437 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.437 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.437 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:20:03.437 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6809", 2026-02-17T21:20:03.437 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:20:03.437 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.438 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.438 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.438 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:20:03.438 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:20:03.438 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.438 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:20:03.438 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6806", 2026-02-17T21:20:03.438 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:20:03.438 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.438 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.438 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:20:03.438 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6807", 2026-02-17T21:20:03.438 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1153784009 2026-02-17T21:20:03.439 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.439 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.439 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.439 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6803/1153784009", 2026-02-17T21:20:03.439 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6805/1153784009", 2026-02-17T21:20:03.439 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6809/1153784009", 2026-02-17T21:20:03.439 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6807/1153784009", 2026-02-17T21:20:03.439 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:20:03.439 INFO:tasks.workunit.client.0.trial031.stdout: "exists", 2026-02-17T21:20:03.439 INFO:tasks.workunit.client.0.trial031.stdout: "up" 2026-02-17T21:20:03.439 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.439 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.440 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.440 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:20:03.440 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "d8cc5e2c-83e0-49d6-8a94-49a26ec2d2cb", 2026-02-17T21:20:03.440 INFO:tasks.workunit.client.0.trial031.stdout: "up": 0, 2026-02-17T21:20:03.440 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:20:03.440 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:20:03.440 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:20:03.440 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 0, 2026-02-17T21:20:03.440 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 0, 2026-02-17T21:20:03.440 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 9, 2026-02-17T21:20:03.440 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 22, 2026-02-17T21:20:03.440 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 30, 2026-02-17T21:20:03.440 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:20:03.440 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:20:03.441 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:20:03.441 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.441 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:20:03.441 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6810", 2026-02-17T21:20:03.441 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:20:03.441 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.441 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.441 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:20:03.441 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6811", 2026-02-17T21:20:03.441 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:20:03.441 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.441 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.442 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.442 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:20:03.442 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:20:03.442 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.442 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:20:03.442 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6812", 2026-02-17T21:20:03.442 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:20:03.442 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.442 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.442 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:20:03.442 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6813", 2026-02-17T21:20:03.442 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:20:03.442 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.442 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.443 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.443 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:20:03.443 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:20:03.443 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.443 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:20:03.443 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6816", 2026-02-17T21:20:03.443 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:20:03.443 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.443 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.443 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:20:03.443 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6817", 2026-02-17T21:20:03.443 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:20:03.444 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.444 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.444 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.444 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:20:03.444 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:20:03.444 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.444 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:20:03.444 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6814", 2026-02-17T21:20:03.444 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:20:03.444 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.444 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.444 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:20:03.444 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6815", 2026-02-17T21:20:03.445 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2990836993 2026-02-17T21:20:03.445 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.445 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.445 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.445 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6811/2990836993", 2026-02-17T21:20:03.445 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6813/2990836993", 2026-02-17T21:20:03.445 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2990836993", 2026-02-17T21:20:03.445 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2990836993", 2026-02-17T21:20:03.445 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:20:03.445 INFO:tasks.workunit.client.0.trial031.stdout: "exists" 2026-02-17T21:20:03.445 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.445 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.445 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.445 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:20:03.446 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "cd50e885-11ef-49a7-a859-6f1549a4e9de", 2026-02-17T21:20:03.446 INFO:tasks.workunit.client.0.trial031.stdout: "up": 1, 2026-02-17T21:20:03.446 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:20:03.446 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:20:03.446 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:20:03.446 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 14, 2026-02-17T21:20:03.446 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 28, 2026-02-17T21:20:03.446 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 38, 2026-02-17T21:20:03.446 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 0, 2026-02-17T21:20:03.446 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 31, 2026-02-17T21:20:03.446 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:20:03.447 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:20:03.447 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:20:03.447 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.447 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:20:03.447 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6810", 2026-02-17T21:20:03.447 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:20:03.447 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.447 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.447 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:20:03.447 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6811", 2026-02-17T21:20:03.447 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:20:03.447 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.447 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.448 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.448 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:20:03.448 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:20:03.448 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.448 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:20:03.448 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6812", 2026-02-17T21:20:03.448 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:20:03.448 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.448 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.448 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:20:03.448 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6813", 2026-02-17T21:20:03.448 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:20:03.448 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.449 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.449 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.449 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:20:03.449 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:20:03.449 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.449 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:20:03.449 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6816", 2026-02-17T21:20:03.449 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:20:03.449 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.449 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.449 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:20:03.449 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6817", 2026-02-17T21:20:03.449 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:20:03.449 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.450 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.450 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.450 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:20:03.450 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:20:03.450 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.450 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:20:03.450 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6814", 2026-02-17T21:20:03.450 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:20:03.450 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.450 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.450 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:20:03.450 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6815", 2026-02-17T21:20:03.450 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2603940079 2026-02-17T21:20:03.451 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.451 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.451 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.451 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6811/2603940079", 2026-02-17T21:20:03.451 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6813/2603940079", 2026-02-17T21:20:03.451 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2603940079", 2026-02-17T21:20:03.451 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2603940079", 2026-02-17T21:20:03.451 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:20:03.451 INFO:tasks.workunit.client.0.trial031.stdout: "exists", 2026-02-17T21:20:03.451 INFO:tasks.workunit.client.0.trial031.stdout: "up" 2026-02-17T21:20:03.451 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.451 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.451 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.452 INFO:tasks.workunit.client.0.trial031.stdout: "osd_xinfo": [ 2026-02-17T21:20:03.452 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.452 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:20:03.452 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:18:51.522342+0000", 2026-02-17T21:20:03.452 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:20:03.452 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:20:03.452 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:20:03.452 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:20:03.452 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "2026-02-17T21:18:07.038217+0000", 2026-02-17T21:20:03.452 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:20:03.452 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.452 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.452 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:20:03.452 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:18:51.801948+0000", 2026-02-17T21:20:03.452 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:20:03.453 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:20:03.453 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:20:03.453 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:20:03.453 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "2026-02-17T21:18:16.051674+0000", 2026-02-17T21:20:03.453 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:20:03.453 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.453 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.453 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:20:03.453 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:18:52.105412+0000", 2026-02-17T21:20:03.453 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:20:03.453 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:20:03.453 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:20:03.453 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:20:03.453 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "2026-02-17T21:18:25.422918+0000", 2026-02-17T21:20:03.454 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:20:03.454 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.454 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.454 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap": [], 2026-02-17T21:20:03.454 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap_items": [], 2026-02-17T21:20:03.454 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap_primaries": [], 2026-02-17T21:20:03.454 INFO:tasks.workunit.client.0.trial031.stdout: "pg_temp": [], 2026-02-17T21:20:03.454 INFO:tasks.workunit.client.0.trial031.stdout: "primary_temp": [], 2026-02-17T21:20:03.454 INFO:tasks.workunit.client.0.trial031.stdout: "blocklist": {}, 2026-02-17T21:20:03.454 INFO:tasks.workunit.client.0.trial031.stdout: "range_blocklist": {}, 2026-02-17T21:20:03.454 INFO:tasks.workunit.client.0.trial031.stdout: "erasure_code_profiles": { 2026-02-17T21:20:03.454 INFO:tasks.workunit.client.0.trial031.stdout: "default": { 2026-02-17T21:20:03.454 INFO:tasks.workunit.client.0.trial031.stdout: "k": "2", 2026-02-17T21:20:03.454 INFO:tasks.workunit.client.0.trial031.stdout: "m": "2", 2026-02-17T21:20:03.454 INFO:tasks.workunit.client.0.trial031.stdout: "plugin": "jerasure", 2026-02-17T21:20:03.455 INFO:tasks.workunit.client.0.trial031.stdout: "technique": "reed_sol_van" 2026-02-17T21:20:03.455 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.455 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.455 INFO:tasks.workunit.client.0.trial031.stdout: "removed_snaps_queue": [], 2026-02-17T21:20:03.455 INFO:tasks.workunit.client.0.trial031.stdout: "new_removed_snaps": [], 2026-02-17T21:20:03.455 INFO:tasks.workunit.client.0.trial031.stdout: "new_purged_snaps": [], 2026-02-17T21:20:03.455 INFO:tasks.workunit.client.0.trial031.stdout: "crush_node_flags": {}, 2026-02-17T21:20:03.455 INFO:tasks.workunit.client.0.trial031.stdout: "device_class_flags": {}, 2026-02-17T21:20:03.455 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode": { 2026-02-17T21:20:03.455 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode_enabled": false, 2026-02-17T21:20:03.455 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_bucket_count": 0, 2026-02-17T21:20:03.455 INFO:tasks.workunit.client.0.trial031.stdout: "degraded_stretch_mode": 0, 2026-02-17T21:20:03.455 INFO:tasks.workunit.client.0.trial031.stdout: "recovering_stretch_mode": 0, 2026-02-17T21:20:03.455 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode_bucket": 0 2026-02-17T21:20:03.455 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: "osd_metadata": [ 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6804/1153784009,v1:127.0.0.1:6805/1153784009]", 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:20:03.456 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:18:00.442071Z", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:20:03.457 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6802/1153784009,v1:127.0.0.1:6803/1153784009]", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/1153784009,v1:127.0.0.1:6809/1153784009]", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/1153784009,v1:127.0.0.1:6807/1153784009]", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:20:03.458 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/0", 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6812/2990836993,v1:127.0.0.1:6813/2990836993]", 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:20:03.459 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:18:09.633013Z", 2026-02-17T21:20:03.460 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6810/2990836993,v1:127.0.0.1:6811/2990836993]", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2990836993,v1:127.0.0.1:6817/2990836993]", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2990836993,v1:127.0.0.1:6815/2990836993]", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:20:03.461 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:20:03.462 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:20:03.462 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:20:03.462 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:20:03.462 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/1", 2026-02-17T21:20:03.462 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:20:03.462 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:20:03.462 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:20:03.462 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.462 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.462 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:20:03.463 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:20:03.463 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6812/2603940079,v1:127.0.0.1:6813/2603940079]", 2026-02-17T21:20:03.463 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:20:03.463 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:20:03.463 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:20:03.463 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:20:03.463 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:20:03.463 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:20:03.464 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:20:03.464 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:20:03.464 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:20:03.464 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:20:03.464 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:20:03.464 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:20:03.464 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-17T21:20:03.464 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:20:03.464 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:20:03.465 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:20:03.465 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:20:03.465 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:20:03.465 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:20:03.465 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:20:03.465 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:20:03.465 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:20:03.465 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:20:03.465 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:18:18.880812Z", 2026-02-17T21:20:03.466 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:20:03.466 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:20:03.466 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:20:03.466 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:20:03.466 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:20:03.466 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:20:03.466 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:20:03.466 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6810/2603940079,v1:127.0.0.1:6811/2603940079]", 2026-02-17T21:20:03.466 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:20:03.467 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2603940079,v1:127.0.0.1:6817/2603940079]", 2026-02-17T21:20:03.467 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2603940079,v1:127.0.0.1:6815/2603940079]", 2026-02-17T21:20:03.467 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:20:03.467 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:20:03.467 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:20:03.467 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:20:03.467 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:20:03.467 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:20:03.467 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:20:03.468 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:20:03.468 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:20:03.468 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:20:03.468 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/2", 2026-02-17T21:20:03.468 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:20:03.468 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:20:03.468 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:20:03.468 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.468 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.468 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_clean_epochs": { 2026-02-17T21:20:03.468 INFO:tasks.workunit.client.0.trial031.stdout: "min_last_epoch_clean": 0, 2026-02-17T21:20:03.468 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_clean": { 2026-02-17T21:20:03.469 INFO:tasks.workunit.client.0.trial031.stdout: "per_pool": [] 2026-02-17T21:20:03.469 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.469 INFO:tasks.workunit.client.0.trial031.stdout: "osd_epochs": [ 2026-02-17T21:20:03.469 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.469 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:20:03.469 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 35 2026-02-17T21:20:03.469 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.469 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.469 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:20:03.469 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 40 2026-02-17T21:20:03.469 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_first_committed": 1, 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_last_committed": 43, 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: "crushmap": { 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: "devices": [ 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.0", 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.1", 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:20:03.470 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.2", 2026-02-17T21:20:03.471 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:20:03.471 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.471 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.471 INFO:tasks.workunit.client.0.trial031.stdout: "types": [ 2026-02-17T21:20:03.471 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.471 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 0, 2026-02-17T21:20:03.471 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd" 2026-02-17T21:20:03.471 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.471 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.471 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:20:03.471 INFO:tasks.workunit.client.0.trial031.stdout: "name": "host" 2026-02-17T21:20:03.471 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.471 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 2, 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: "name": "chassis" 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 3, 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: "name": "rack" 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 4, 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: "name": "row" 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 5, 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: "name": "pdu" 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.472 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 6, 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: "name": "pod" 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 7, 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: "name": "room" 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 8, 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: "name": "datacenter" 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 9, 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: "name": "zone" 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 10, 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: "name": "region" 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:20:03.473 INFO:tasks.workunit.client.0.trial031.stdout: "name": "root" 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "buckets": [ 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "id": -1, 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "name": "default", 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "root", 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "id": -3, 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "id": -2, 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "name": "default~ssd", 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "root", 2026-02-17T21:20:03.474 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "id": -4, 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "id": -3, 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "name": "trial031", 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "host", 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.475 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 1 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 2 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "id": -4, 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "name": "trial031~ssd", 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "host", 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.476 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 1 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 2 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "rules": [ 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "rule_id": 0, 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "rule_name": "replicated_rule", 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "type": 1, 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "steps": [ 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "op": "take", 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "item": -1, 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "item_name": "default" 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "op": "choose_firstn", 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "num": 0, 2026-02-17T21:20:03.477 INFO:tasks.workunit.client.0.trial031.stdout: "type": "osd" 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "op": "emit" 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "tunables": { 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "choose_local_tries": 0, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "choose_local_fallback_tries": 0, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "choose_total_tries": 50, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_descend_once": 1, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_vary_r": 1, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_stable": 1, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "msr_descents": 100, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "msr_collision_tries": 100, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "straw_calc_version": 1, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "allowed_bucket_algs": 54, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "profile": "jewel", 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "optimal_tunables": 1, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "legacy_tunables": 0, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "minimum_required_version": "jewel", 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables": 1, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables2": 1, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "has_v2_rules": 0, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables3": 1, 2026-02-17T21:20:03.478 INFO:tasks.workunit.client.0.trial031.stdout: "has_v3_rules": 0, 2026-02-17T21:20:03.479 INFO:tasks.workunit.client.0.trial031.stderr:report 2711718807 2026-02-17T21:20:03.479 INFO:tasks.workunit.client.0.trial031.stdout: "has_v4_buckets": 1, 2026-02-17T21:20:03.479 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables5": 1, 2026-02-17T21:20:03.479 INFO:tasks.workunit.client.0.trial031.stdout: "has_v5_rules": 0, 2026-02-17T21:20:03.479 INFO:tasks.workunit.client.0.trial031.stdout: "has_msr_rules": 0 2026-02-17T21:20:03.479 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.479 INFO:tasks.workunit.client.0.trial031.stdout: "choose_args": {} 2026-02-17T21:20:03.479 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.479 INFO:tasks.workunit.client.0.trial031.stdout: "fsmap": { 2026-02-17T21:20:03.479 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 1, 2026-02-17T21:20:03.479 INFO:tasks.workunit.client.0.trial031.stdout: "btime": "2026-02-17T21:17:57:753188+0000", 2026-02-17T21:20:03.479 INFO:tasks.workunit.client.0.trial031.stdout: "default_fscid": -1, 2026-02-17T21:20:03.479 INFO:tasks.workunit.client.0.trial031.stdout: "compat": { 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "compat": {}, 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "ro_compat": {}, 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "incompat": { 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "feature_1": "base v0.20", 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "feature_2": "client writeable ranges", 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "feature_3": "default file layouts on dirs", 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "feature_4": "dir inode in separate object", 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "feature_5": "mds uses versioned encoding", 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "feature_8": "no anchor table", 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "feature_9": "file layout v2", 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "feature_10": "snaprealm v2", 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "feature_11": "minor log segments", 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "feature_12": "quiesce subvolumes" 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "feature_flags": { 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "enable_multiple": true, 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "ever_enabled_multiple": true 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "standbys": [], 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "filesystems": [] 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "mdsmap_first_committed": 1, 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "mdsmap_last_committed": 1, 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "auth": { 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "first_committed": 1, 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "last_committed": 5, 2026-02-17T21:20:03.480 INFO:tasks.workunit.client.0.trial031.stdout: "num_secrets": 3 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg": 1, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_active": 1, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_unknown": 0, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_osd": 3, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "pool_sum": { 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "stat_sum": { 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes": 70861, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects": 100, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_clones": 0, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_copies": 300, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing": 0, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_degraded": 100, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_misplaced": 0, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_unfound": 0, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_dirty": 100, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_whiteouts": 0, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_read": 0, 2026-02-17T21:20:03.481 INFO:tasks.workunit.client.0.trial031.stdout: "num_read_kb": 0, 2026-02-17T21:20:03.482 INFO:tasks.workunit.client.0.trial031.stdout: "num_write": 101, 2026-02-17T21:20:03.482 INFO:tasks.workunit.client.0.trial031.stdout: "num_write_kb": 101, 2026-02-17T21:20:03.482 INFO:tasks.workunit.client.0.trial031.stdout: "num_scrub_errors": 0, 2026-02-17T21:20:03.482 INFO:tasks.workunit.client.0.trial031.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T21:20:03.482 INFO:tasks.workunit.client.0.trial031.stdout: "num_deep_scrub_errors": 0, 2026-02-17T21:20:03.482 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_recovered": 0, 2026-02-17T21:20:03.482 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_recovered": 0, 2026-02-17T21:20:03.482 INFO:tasks.workunit.client.0.trial031.stdout: "num_keys_recovered": 0, 2026-02-17T21:20:03.482 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_omap": 0, 2026-02-17T21:20:03.482 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T21:20:03.482 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T21:20:03.482 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush": 0, 2026-02-17T21:20:03.482 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_kb": 0, 2026-02-17T21:20:03.482 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict": 0, 2026-02-17T21:20:03.482 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_kb": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "num_promote": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_high": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_low": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_some": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_full": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_pinned": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "num_legacy_snapsets": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "num_large_omap_objects": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_manifest": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_bytes": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_keys": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_repaired": 0 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "store_stats": { 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "total": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "available": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:20:03.483 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 0, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 0 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "log_size": 17, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "ondisk_log_size": 17, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "up": 2, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "acting": 2, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "num_store_stats": 0 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "osd_sum": { 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 0, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 0, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "num_pgs": 3, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "num_osds": 3, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_osds": 3, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "kb": 314572800, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used": 83956, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_data": 2128, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_omap": 10, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_meta": 81717, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "kb_avail": 314488844, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "statfs": { 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "total": 322122547200, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "available": 322036576256, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 2179072, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 498158, 2026-02-17T21:20:03.484 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 10389, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 83679083 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "hb_peers": [], 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "snap_trim_queue_len": 0, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "num_snap_trimming": 0, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "num_shards_repaired": 0, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "op_queue_age_hist": { 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "histogram": [], 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "upper_bound": 1 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "perf_stat": { 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ms": 0, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ms": 0, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ns": 0, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ns": 0 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "alerts": [], 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "network_ping_times": [] 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "osd_sum_by_class": { 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "ssd": { 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 0, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 0, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "num_pgs": 3, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "num_osds": 3, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_osds": 3, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T21:20:03.485 INFO:tasks.workunit.client.0.trial031.stdout: "kb": 314572800, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used": 83956, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_data": 2128, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_omap": 10, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_meta": 81717, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "kb_avail": 314488844, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "statfs": { 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "total": 322122547200, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "available": 322036576256, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 2179072, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 498158, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 10389, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 83679083 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "hb_peers": [], 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "snap_trim_queue_len": 0, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "num_snap_trimming": 0, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "num_shards_repaired": 0, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "op_queue_age_hist": { 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "histogram": [], 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "upper_bound": 1 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "perf_stat": { 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ms": 0, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ms": 0, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ns": 0, 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ns": 0 2026-02-17T21:20:03.486 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "alerts": [], 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "network_ping_times": [] 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "pool_stats": [ 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "poolid": 1, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg": 1, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "stat_sum": { 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes": 70861, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects": 100, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_clones": 0, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_copies": 300, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing": 0, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_degraded": 100, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_misplaced": 0, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_unfound": 0, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_dirty": 100, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_whiteouts": 0, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_read": 0, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_read_kb": 0, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_write": 101, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_write_kb": 101, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_scrub_errors": 0, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_deep_scrub_errors": 0, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_recovered": 0, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_recovered": 0, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_keys_recovered": 0, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_omap": 0, 2026-02-17T21:20:03.487 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_kb": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_kb": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_promote": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_high": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_low": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_some": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_full": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_pinned": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_legacy_snapsets": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_large_omap_objects": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_manifest": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_bytes": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_keys": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_repaired": 0 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "store_stats": { 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "total": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "available": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 1220608, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 211432, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 0, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 0 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.488 INFO:tasks.workunit.client.0.trial031.stdout: "log_size": 17, 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "ondisk_log_size": 17, 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "up": 2, 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "acting": 2, 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "num_store_stats": 3 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "osd_stats": [ 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 150323855377 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 38654705677 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 163208757264 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_by_state": [ 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "state": "active+undersized+degraded", 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "num": 1 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_by_osd": [ 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:20:03.489 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 1, 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 1, 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 1, 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 0, 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snaps": [ 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "pool": 1, 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snaps": {} 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "servicemap": { 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 5, 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:20:00.202016+0000", 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "services": { 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "osd": { 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "daemons": { 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "summary": "", 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "2": { 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "start_epoch": 0, 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "start_stamp": "0.000000", 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "gid": 0, 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-17T21:20:03.490 INFO:tasks.workunit.client.0.trial031.stdout: "metadata": {}, 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: "task_status": {} 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: "mgrstat_first_committed": 1, 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: "mgrstat_last_committed": 60, 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: "logm_first_committed": 1, 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: "logm_last_committed": 291, 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: "paxos": { 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: "first_committed": 1, 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: "last_committed": 405, 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: "last_pn": 0, 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: "accepted_pn": 0 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:536: TEST_divergent_2: echo 'delay recovery' 2026-02-17T21:20:03.491 INFO:tasks.workunit.client.0.trial031.stdout:delay recovery 2026-02-17T21:20:03.492 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:537: TEST_divergent_2: ceph pg dump pgs 2026-02-17T21:20:03.627 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:20:03.627 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70861 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:19:19.818123+0000 43'101 43:136 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:18:27.226024+0000 0'0 2026-02-17T21:18:27.226024+0000 0 0 periodic scrub scheduled @ 2026-02-19T05:02:25.367558+0000 0 0 2026-02-17T21:20:03.627 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:20:03.627 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:20:03.627 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:20:03.635 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:538: TEST_divergent_2: for i in $non_divergent 2026-02-17T21:20:03.636 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: get_asok_path osd.0 2026-02-17T21:20:03.636 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T21:20:03.636 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T21:20:03.637 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:20:03.637 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:20:03.637 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:20:03.637 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.0.asok 2026-02-17T21:20:03.637 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: CEPH_ARGS= 2026-02-17T21:20:03.637 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.0.asok set_recovery_delay 100000 2026-02-17T21:20:03.699 INFO:tasks.workunit.client.0.trial031.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-17T21:20:03.705 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:538: TEST_divergent_2: for i in $non_divergent 2026-02-17T21:20:03.706 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: get_asok_path osd.2 2026-02-17T21:20:03.706 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-17T21:20:03.706 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-17T21:20:03.706 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:20:03.707 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:20:03.707 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:20:03.708 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.2.asok 2026-02-17T21:20:03.708 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: CEPH_ARGS= 2026-02-17T21:20:03.708 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.2.asok set_recovery_delay 100000 2026-02-17T21:20:03.762 INFO:tasks.workunit.client.0.trial031.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-17T21:20:03.768 INFO:tasks.workunit.client.0.trial031.stdout:revive divergent 1 2026-02-17T21:20:03.768 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:544: TEST_divergent_2: echo 'revive divergent 1' 2026-02-17T21:20:03.768 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:545: TEST_divergent_2: ceph pg dump pgs 2026-02-17T21:20:03.972 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:20:03.972 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70861 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:19:19.818123+0000 43'101 43:136 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:18:27.226024+0000 0'0 2026-02-17T21:18:27.226024+0000 0 0 periodic scrub scheduled @ 2026-02-19T05:02:25.367558+0000 0 0 2026-02-17T21:20:03.972 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:20:03.972 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:20:03.973 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:20:03.980 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:546: TEST_divergent_2: ceph osd set noup 2026-02-17T21:20:04.300 INFO:tasks.workunit.client.0.trial031.stderr:noup is set 2026-02-17T21:20:04.316 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:547: TEST_divergent_2: activate_osd td/divergent-priors 1 2026-02-17T21:20:04.316 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:20:04.316 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:20:04.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T21:20:04.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:20:04.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-17T21:20:04.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:20:04.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:20:04.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:20:04.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:20:04.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-17T21:20:04.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-17T21:20:04.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:20:04.318 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:20:04.318 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:20:04.318 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:20:04.318 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:20:04.318 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:20:04.318 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:20:04.318 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:20:04.318 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:20:04.318 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:20:04.318 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:20:04.318 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:20:04.318 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:20:04.318 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:20:04.319 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:20:04.319 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:20:04.319 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:20:04.319 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:20:04.319 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:20:04.319 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:20:04.319 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-17T21:20:04.320 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:20:04.320 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T21:20:04.321 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:20:04.321 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-17T21:20:04.321 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T21:20:04.321 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:20:04.321 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:20:04.321 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:20:04.338 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:04.335+0000 7fcdc41638c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:04.339 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:04.339+0000 7fcdc41638c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:04.340 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:04.339+0000 7fcdc41638c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:04.518 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:880: activate_osd: return 0 2026-02-17T21:20:04.518 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:548: TEST_divergent_2: sleep 5 2026-02-17T21:20:05.560 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:05.559+0000 7fcdc41638c0 -1 Falling back to public interface 2026-02-17T21:20:08.521 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:08.519+0000 7fcdc41638c0 -1 osd.1 28 log_to_monitors true 2026-02-17T21:20:09.520 INFO:tasks.workunit.client.0.trial031.stdout:delay recovery divergent 2026-02-17T21:20:09.520 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:550: TEST_divergent_2: echo 'delay recovery divergent' 2026-02-17T21:20:09.520 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:551: TEST_divergent_2: ceph pg dump pgs 2026-02-17T21:20:09.712 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:20:09.712 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70861 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:19:19.818123+0000 43'101 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:18:27.226024+0000 0'0 2026-02-17T21:18:27.226024+0000 0 0 periodic scrub scheduled @ 2026-02-19T05:02:25.367558+0000 0 0 2026-02-17T21:20:09.712 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:20:09.712 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:20:09.712 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:20:09.721 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:552: TEST_divergent_2: get_asok_path osd.1 2026-02-17T21:20:09.721 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T21:20:09.721 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T21:20:09.721 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:20:09.721 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:20:09.721 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:20:09.721 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.1.asok 2026-02-17T21:20:09.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:552: TEST_divergent_2: CEPH_ARGS= 2026-02-17T21:20:09.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:552: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.1.asok set_recovery_delay 100000 2026-02-17T21:20:09.776 INFO:tasks.workunit.client.0.trial031.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-17T21:20:09.782 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:554: TEST_divergent_2: ceph osd unset noup 2026-02-17T21:20:10.095 INFO:tasks.workunit.client.0.trial031.stderr:noup is unset 2026-02-17T21:20:10.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:556: TEST_divergent_2: wait_for_osd up 0 2026-02-17T21:20:10.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:20:10.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:20:10.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:20:10.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:20:10.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:20:10.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:20:10.108 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:20:10.109 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:20:10.109 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:20:10.315 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 35 up_thru 42 down_at 29 last_clean_interval [5,28) [v2:127.0.0.1:6802/1153784009,v1:127.0.0.1:6803/1153784009] [v2:127.0.0.1:6804/1153784009,v1:127.0.0.1:6805/1153784009] exists,up 505cc7f4-fb42-4465-8933-5f18295caf89 2026-02-17T21:20:10.316 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:20:10.316 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:20:10.316 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:20:10.316 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:557: TEST_divergent_2: wait_for_osd up 1 2026-02-17T21:20:10.316 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:20:10.316 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:20:10.316 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:20:10.316 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:20:10.316 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:20:10.317 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:20:10.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:20:10.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:20:10.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:20:10.514 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 48 up_thru 48 down_at 30 last_clean_interval [9,28) [v2:127.0.0.1:6818/2358018890,v1:127.0.0.1:6819/2358018890] [v2:127.0.0.1:6820/2358018890,v1:127.0.0.1:6821/2358018890] exists,up d8cc5e2c-83e0-49d6-8a94-49a26ec2d2cb 2026-02-17T21:20:10.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:20:10.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:20:10.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:20:10.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:558: TEST_divergent_2: wait_for_osd up 2 2026-02-17T21:20:10.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:20:10.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:20:10.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:20:10.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:20:10.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:20:10.517 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:20:10.517 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:20:10.517 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:20:10.517 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:20:10.717 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 38 up_thru 0 down_at 31 last_clean_interval [14,28) [v2:127.0.0.1:6810/2603940079,v1:127.0.0.1:6811/2603940079] [v2:127.0.0.1:6812/2603940079,v1:127.0.0.1:6813/2603940079] exists,up cd50e885-11ef-49a7-a859-6f1549a4e9de 2026-02-17T21:20:10.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:20:10.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:20:10.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:20:10.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:560: TEST_divergent_2: ceph pg dump pgs 2026-02-17T21:20:10.907 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:20:10.907 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70861 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:19:19.818123+0000 43'101 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:18:27.226024+0000 0'0 2026-02-17T21:18:27.226024+0000 0 0 periodic scrub scheduled @ 2026-02-19T05:02:25.367558+0000 0 0 2026-02-17T21:20:10.908 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:20:10.908 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:20:10.908 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:20:10.916 INFO:tasks.workunit.client.0.trial031.stdout:wait for peering 2026-02-17T21:20:10.916 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:561: TEST_divergent_2: echo 'wait for peering' 2026-02-17T21:20:10.916 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:562: TEST_divergent_2: ceph pg dump pgs 2026-02-17T21:20:11.122 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:20:11.122 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70861 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:19:19.818123+0000 43'101 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:18:27.226024+0000 0'0 2026-02-17T21:18:27.226024+0000 0 0 periodic scrub scheduled @ 2026-02-19T05:02:25.367558+0000 0 0 2026-02-17T21:20:11.123 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:20:11.123 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:20:11.123 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:20:11.130 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:563: TEST_divergent_2: rados -p test put foo /etc/fstab 2026-02-17T21:20:11.161 INFO:tasks.workunit.client.0.trial031.stdout:killing divergent 1 2026-02-17T21:20:11.161 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:567: TEST_divergent_2: echo 'killing divergent 1' 2026-02-17T21:20:11.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:568: TEST_divergent_2: ceph pg dump pgs 2026-02-17T21:20:11.354 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:20:11.354 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70861 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:19:19.818123+0000 43'101 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:18:27.226024+0000 0'0 2026-02-17T21:18:27.226024+0000 0 0 periodic scrub scheduled @ 2026-02-19T05:02:25.367558+0000 0 0 2026-02-17T21:20:11.354 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:20:11.354 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:20:11.355 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:20:11.362 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:569: TEST_divergent_2: kill_daemons td/divergent-priors KILL osd.1 2026-02-17T21:20:11.363 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:20:11.363 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:20:11.363 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:20:11.363 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:20:11.364 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:20:11.471 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:20:11.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:572: TEST_divergent_2: expfile=td/divergent-priors/exp.16076.out 2026-02-17T21:20:11.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:573: TEST_divergent_2: _objectstore_tool_nodown td/divergent-priors 1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.16076.out 2026-02-17T21:20:11.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-17T21:20:11.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T21:20:11.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T21:20:11.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T21:20:11.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-17T21:20:11.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.16076.out 2026-02-17T21:20:12.112 INFO:tasks.workunit.client.0.trial031.stderr:Exporting 1.0 info 1.0( v 49'102 lc 0'0 (28'90,49'102] local-lis/les=48/49 n=101 ec=22/22 lis/c=48/22 les/c/f=49/27/0 sis=48) 2026-02-17T21:20:12.113 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:023dd3d2:::existing_12:head# 2026-02-17T21:20:12.113 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:04c40a0c:::existing_93:head# 2026-02-17T21:20:12.113 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:064ff5fb:::existing_36:head# 2026-02-17T21:20:12.114 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:091f9f1d:::existing_94:head# 2026-02-17T21:20:12.114 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:0a865a93:::existing_54:head# 2026-02-17T21:20:12.114 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:0fed513c:::existing_41:head# 2026-02-17T21:20:12.114 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:13b9e0b8:::existing_92:head# 2026-02-17T21:20:12.114 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:169157fa:::existing_81:head# 2026-02-17T21:20:12.114 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:17731382:::existing_58:head# 2026-02-17T21:20:12.115 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:18a45a39:::existing_43:head# 2026-02-17T21:20:12.115 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:1c160f01:::existing_66:head# 2026-02-17T21:20:12.115 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:1dd6c249:::existing_97:head# 2026-02-17T21:20:12.115 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:227e0c77:::existing_7:head# 2026-02-17T21:20:12.116 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:23e1b063:::existing_59:head# 2026-02-17T21:20:12.116 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:28566edb:::existing_78:head# 2026-02-17T21:20:12.116 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:2a101182:::existing_20:head# 2026-02-17T21:20:12.116 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:2d1519da:::existing_96:head# 2026-02-17T21:20:12.116 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:3078b87a:::existing_86:head# 2026-02-17T21:20:12.116 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:30bde01b:::existing_15:head# 2026-02-17T21:20:12.117 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:317a7e19:::existing_56:head# 2026-02-17T21:20:12.117 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:38b9c7fa:::existing_33:head# 2026-02-17T21:20:12.117 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:3e16d84b:::existing_48:head# 2026-02-17T21:20:12.117 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:45fe8203:::existing_75:head# 2026-02-17T21:20:12.117 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:4e82c9ba:::existing_50:head# 2026-02-17T21:20:12.117 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:5298dcdb:::existing_57:head# 2026-02-17T21:20:12.117 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:52ebaa44:::existing_32:head# 2026-02-17T21:20:12.118 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:53588912:::existing_9:head# 2026-02-17T21:20:12.118 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:5b7e2de6:::existing_84:head# 2026-02-17T21:20:12.118 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:5da0255c:::existing_13:head# 2026-02-17T21:20:12.118 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:5da2dab8:::existing_53:head# 2026-02-17T21:20:12.118 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:5ee3514b:::existing_30:head# 2026-02-17T21:20:12.119 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:602f83fe:::foo:head# 2026-02-17T21:20:12.119 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:60426cc7:::existing_62:head# 2026-02-17T21:20:12.119 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:628918ca:::existing_79:head# 2026-02-17T21:20:12.119 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:63f8ab2d:::existing_77:head# 2026-02-17T21:20:12.120 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:67e1780f:::existing_27:head# 2026-02-17T21:20:12.120 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:68143ca1:::existing_71:head# 2026-02-17T21:20:12.120 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:6bac9509:::existing_29:head# 2026-02-17T21:20:12.120 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:6bbec798:::existing_45:head# 2026-02-17T21:20:12.120 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:6cb23a40:::existing_65:head# 2026-02-17T21:20:12.120 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:7055ec5e:::existing_83:head# 2026-02-17T21:20:12.120 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:70a54e2b:::existing_87:head# 2026-02-17T21:20:12.120 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:750ace21:::existing_19:head# 2026-02-17T21:20:12.121 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:78623b1d:::existing_6:head# 2026-02-17T21:20:12.121 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:79881152:::existing_63:head# 2026-02-17T21:20:12.121 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:7a80c0d5:::existing_99:head# 2026-02-17T21:20:12.121 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:81490677:::existing_76:head# 2026-02-17T21:20:12.121 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:85ec8e5d:::existing_85:head# 2026-02-17T21:20:12.122 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:8e2abbf0:::existing_49:head# 2026-02-17T21:20:12.122 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:8ee0fe0f:::existing_52:head# 2026-02-17T21:20:12.122 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:923c5d4d:::existing_5:head# 2026-02-17T21:20:12.122 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:92731e71:::existing_89:head# 2026-02-17T21:20:12.123 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:928205cf:::existing_82:head# 2026-02-17T21:20:12.123 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:93be2267:::existing_24:head# 2026-02-17T21:20:12.123 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:99ceda2c:::existing_44:head# 2026-02-17T21:20:12.123 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:9dd917ee:::existing_26:head# 2026-02-17T21:20:12.123 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:a28805a6:::existing_72:head# 2026-02-17T21:20:12.123 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:a294a8f6:::existing_16:head# 2026-02-17T21:20:12.123 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:a2f5473a:::existing_42:head# 2026-02-17T21:20:12.123 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:a504a533:::existing_90:head# 2026-02-17T21:20:12.123 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:a691337e:::existing_61:head# 2026-02-17T21:20:12.124 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:a6e8d7ac:::existing_28:head# 2026-02-17T21:20:12.124 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:a9fde86e:::existing_18:head# 2026-02-17T21:20:12.124 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:aa309faa:::existing_68:head# 2026-02-17T21:20:12.124 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:ae40b0ed:::existing_31:head# 2026-02-17T21:20:12.124 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:b279b1fe:::existing_10:head# 2026-02-17T21:20:12.125 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:b343e277:::existing_88:head# 2026-02-17T21:20:12.125 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:b7130e90:::existing_80:head# 2026-02-17T21:20:12.125 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:bc6f7309:::existing_70:head# 2026-02-17T21:20:12.125 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:be8489b7:::existing_38:head# 2026-02-17T21:20:12.126 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:beb4e5e3:::existing_47:head# 2026-02-17T21:20:12.126 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:bebec877:::existing_74:head# 2026-02-17T21:20:12.126 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:c04347d8:::existing_60:head# 2026-02-17T21:20:12.126 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:c156033e:::existing_100:head# 2026-02-17T21:20:12.126 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:c19f5a8e:::existing_8:head# 2026-02-17T21:20:12.126 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:c3e6ebe4:::existing_17:head# 2026-02-17T21:20:12.126 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:c6c5a7c2:::existing_39:head# 2026-02-17T21:20:12.126 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:cc1d0abf:::existing_95:head# 2026-02-17T21:20:12.127 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:ccb3865b:::existing_73:head# 2026-02-17T21:20:12.127 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:cd84e19e:::existing_40:head# 2026-02-17T21:20:12.127 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:d2cd64ce:::existing_37:head# 2026-02-17T21:20:12.127 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:d41a25c9:::existing_22:head# 2026-02-17T21:20:12.127 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:d7f088da:::existing_91:head# 2026-02-17T21:20:12.128 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:d9cbd1d6:::existing_64:head# 2026-02-17T21:20:12.128 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:dcc4ff82:::existing_98:head# 2026-02-17T21:20:12.128 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:dd677bd3:::existing_35:head# 2026-02-17T21:20:12.128 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:dd6d90e7:::existing_46:head# 2026-02-17T21:20:12.129 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:e0e22077:::existing_21:head# 2026-02-17T21:20:12.129 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:e255b710:::existing_55:head# 2026-02-17T21:20:12.129 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:eff1d5db:::existing_14:head# 2026-02-17T21:20:12.129 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:f1b156bf:::existing_23:head# 2026-02-17T21:20:12.129 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:f5ce4199:::existing_34:head# 2026-02-17T21:20:12.129 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:f6809456:::existing_11:head# 2026-02-17T21:20:12.129 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:f699a2a9:::existing_67:head# 2026-02-17T21:20:12.129 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:f9baac43:::existing_25:head# 2026-02-17T21:20:12.130 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:fb76688e:::existing_51:head# 2026-02-17T21:20:12.130 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:fbb92eec:::existing_69:head# 2026-02-17T21:20:12.130 INFO:tasks.workunit.client.0.trial031.stderr:Export successful 2026-02-17T21:20:12.130 INFO:tasks.workunit.client.0.trial031.stdout: marking collection for removal 2026-02-17T21:20:12.130 INFO:tasks.workunit.client.0.trial031.stdout:setting '_remove' omap key 2026-02-17T21:20:12.130 INFO:tasks.workunit.client.0.trial031.stdout:finish_remove_pgs 1.0_head removing 1.0 2026-02-17T21:20:12.131 INFO:tasks.workunit.client.0.trial031.stderr:Remove successful 2026-02-17T21:20:12.664 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:574: TEST_divergent_2: _objectstore_tool_nodown td/divergent-priors 1 --op import --file td/divergent-priors/exp.16076.out 2026-02-17T21:20:12.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-17T21:20:12.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T21:20:12.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T21:20:12.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T21:20:12.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-17T21:20:12.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op import --file td/divergent-priors/exp.16076.out 2026-02-17T21:20:13.295 INFO:tasks.workunit.client.0.trial031.stdout:get_pg_num_history pg_num_history pg_num_history(e49 pg_nums {1={22=1}} deleted_pools ) 2026-02-17T21:20:13.295 INFO:tasks.workunit.client.0.trial031.stdout:Importing pgid 1.0 2026-02-17T21:20:13.295 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:023dd3d2:::existing_12:head# 2026-02-17T21:20:13.296 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.298 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:04c40a0c:::existing_93:head# 2026-02-17T21:20:13.298 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.298 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:064ff5fb:::existing_36:head# 2026-02-17T21:20:13.298 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.298 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:091f9f1d:::existing_94:head# 2026-02-17T21:20:13.298 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.298 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:0a865a93:::existing_54:head# 2026-02-17T21:20:13.298 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.298 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:0fed513c:::existing_41:head# 2026-02-17T21:20:13.299 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.299 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:13b9e0b8:::existing_92:head# 2026-02-17T21:20:13.299 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.299 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:169157fa:::existing_81:head# 2026-02-17T21:20:13.299 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.299 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:17731382:::existing_58:head# 2026-02-17T21:20:13.299 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.299 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:18a45a39:::existing_43:head# 2026-02-17T21:20:13.299 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.299 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:1c160f01:::existing_66:head# 2026-02-17T21:20:13.299 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.303 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:1dd6c249:::existing_97:head# 2026-02-17T21:20:13.303 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.303 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:227e0c77:::existing_7:head# 2026-02-17T21:20:13.303 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.303 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:23e1b063:::existing_59:head# 2026-02-17T21:20:13.304 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.304 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:28566edb:::existing_78:head# 2026-02-17T21:20:13.304 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.304 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:2a101182:::existing_20:head# 2026-02-17T21:20:13.304 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.304 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:2d1519da:::existing_96:head# 2026-02-17T21:20:13.304 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.304 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:3078b87a:::existing_86:head# 2026-02-17T21:20:13.304 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.304 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:30bde01b:::existing_15:head# 2026-02-17T21:20:13.304 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.305 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:317a7e19:::existing_56:head# 2026-02-17T21:20:13.305 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.305 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:38b9c7fa:::existing_33:head# 2026-02-17T21:20:13.305 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.305 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:3e16d84b:::existing_48:head# 2026-02-17T21:20:13.305 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.305 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:45fe8203:::existing_75:head# 2026-02-17T21:20:13.305 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.305 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:4e82c9ba:::existing_50:head# 2026-02-17T21:20:13.305 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.306 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:5298dcdb:::existing_57:head# 2026-02-17T21:20:13.306 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.306 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:52ebaa44:::existing_32:head# 2026-02-17T21:20:13.306 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.306 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:53588912:::existing_9:head# 2026-02-17T21:20:13.306 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.306 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:5b7e2de6:::existing_84:head# 2026-02-17T21:20:13.306 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.306 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:5da0255c:::existing_13:head# 2026-02-17T21:20:13.306 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.306 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:5da2dab8:::existing_53:head# 2026-02-17T21:20:13.306 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.306 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:5ee3514b:::existing_30:head# 2026-02-17T21:20:13.307 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.313 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:602f83fe:::foo:head# 2026-02-17T21:20:13.313 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.313 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:60426cc7:::existing_62:head# 2026-02-17T21:20:13.314 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.314 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:628918ca:::existing_79:head# 2026-02-17T21:20:13.314 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.314 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:63f8ab2d:::existing_77:head# 2026-02-17T21:20:13.314 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.314 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:67e1780f:::existing_27:head# 2026-02-17T21:20:13.314 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.314 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:68143ca1:::existing_71:head# 2026-02-17T21:20:13.314 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.314 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:6bac9509:::existing_29:head# 2026-02-17T21:20:13.314 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.314 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:6bbec798:::existing_45:head# 2026-02-17T21:20:13.315 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.315 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:6cb23a40:::existing_65:head# 2026-02-17T21:20:13.315 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.315 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:7055ec5e:::existing_83:head# 2026-02-17T21:20:13.315 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.315 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:70a54e2b:::existing_87:head# 2026-02-17T21:20:13.315 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.315 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:750ace21:::existing_19:head# 2026-02-17T21:20:13.315 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.315 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:78623b1d:::existing_6:head# 2026-02-17T21:20:13.315 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.315 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:79881152:::existing_63:head# 2026-02-17T21:20:13.315 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.316 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:7a80c0d5:::existing_99:head# 2026-02-17T21:20:13.316 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.316 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:81490677:::existing_76:head# 2026-02-17T21:20:13.316 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.316 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:85ec8e5d:::existing_85:head# 2026-02-17T21:20:13.316 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.316 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:8e2abbf0:::existing_49:head# 2026-02-17T21:20:13.316 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.316 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:8ee0fe0f:::existing_52:head# 2026-02-17T21:20:13.316 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.316 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:923c5d4d:::existing_5:head# 2026-02-17T21:20:13.316 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.316 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:92731e71:::existing_89:head# 2026-02-17T21:20:13.317 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.317 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:928205cf:::existing_82:head# 2026-02-17T21:20:13.317 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.317 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:93be2267:::existing_24:head# 2026-02-17T21:20:13.317 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.317 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:99ceda2c:::existing_44:head# 2026-02-17T21:20:13.317 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.317 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:9dd917ee:::existing_26:head# 2026-02-17T21:20:13.317 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.317 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:a28805a6:::existing_72:head# 2026-02-17T21:20:13.317 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.317 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:a294a8f6:::existing_16:head# 2026-02-17T21:20:13.317 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.318 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:a2f5473a:::existing_42:head# 2026-02-17T21:20:13.318 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.318 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:a504a533:::existing_90:head# 2026-02-17T21:20:13.318 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.318 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:a691337e:::existing_61:head# 2026-02-17T21:20:13.318 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.318 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:a6e8d7ac:::existing_28:head# 2026-02-17T21:20:13.318 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.318 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:a9fde86e:::existing_18:head# 2026-02-17T21:20:13.318 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.318 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:aa309faa:::existing_68:head# 2026-02-17T21:20:13.318 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.318 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:ae40b0ed:::existing_31:head# 2026-02-17T21:20:13.319 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.319 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:b279b1fe:::existing_10:head# 2026-02-17T21:20:13.319 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.319 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:b343e277:::existing_88:head# 2026-02-17T21:20:13.319 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.319 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:b7130e90:::existing_80:head# 2026-02-17T21:20:13.319 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.324 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:bc6f7309:::existing_70:head# 2026-02-17T21:20:13.325 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.325 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:be8489b7:::existing_38:head# 2026-02-17T21:20:13.325 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.325 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:beb4e5e3:::existing_47:head# 2026-02-17T21:20:13.325 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.325 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:bebec877:::existing_74:head# 2026-02-17T21:20:13.325 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.325 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:c04347d8:::existing_60:head# 2026-02-17T21:20:13.325 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.325 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:c156033e:::existing_100:head# 2026-02-17T21:20:13.325 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.326 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:c19f5a8e:::existing_8:head# 2026-02-17T21:20:13.326 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.326 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:c3e6ebe4:::existing_17:head# 2026-02-17T21:20:13.326 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.326 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:c6c5a7c2:::existing_39:head# 2026-02-17T21:20:13.326 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.326 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:cc1d0abf:::existing_95:head# 2026-02-17T21:20:13.326 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.326 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:ccb3865b:::existing_73:head# 2026-02-17T21:20:13.326 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.326 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:cd84e19e:::existing_40:head# 2026-02-17T21:20:13.326 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.326 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:d2cd64ce:::existing_37:head# 2026-02-17T21:20:13.327 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.327 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:d41a25c9:::existing_22:head# 2026-02-17T21:20:13.327 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.327 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:d7f088da:::existing_91:head# 2026-02-17T21:20:13.327 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.327 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:d9cbd1d6:::existing_64:head# 2026-02-17T21:20:13.327 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.327 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:dcc4ff82:::existing_98:head# 2026-02-17T21:20:13.327 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.327 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:dd677bd3:::existing_35:head# 2026-02-17T21:20:13.327 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.327 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:dd6d90e7:::existing_46:head# 2026-02-17T21:20:13.327 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.328 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:e0e22077:::existing_21:head# 2026-02-17T21:20:13.328 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.328 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:e255b710:::existing_55:head# 2026-02-17T21:20:13.328 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.328 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:eff1d5db:::existing_14:head# 2026-02-17T21:20:13.328 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.328 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:f1b156bf:::existing_23:head# 2026-02-17T21:20:13.328 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.328 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:f5ce4199:::existing_34:head# 2026-02-17T21:20:13.328 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.328 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:f6809456:::existing_11:head# 2026-02-17T21:20:13.328 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.328 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:f699a2a9:::existing_67:head# 2026-02-17T21:20:13.329 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.329 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:f9baac43:::existing_25:head# 2026-02-17T21:20:13.329 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.329 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:fb76688e:::existing_51:head# 2026-02-17T21:20:13.329 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.329 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:fbb92eec:::existing_69:head# 2026-02-17T21:20:13.329 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:20:13.329 INFO:tasks.workunit.client.0.trial031.stdout:write_pg epoch 49 info 1.0( v 49'102 lc 0'0 (28'90,49'102] local-lis/les=48/49 n=101 ec=22/22 lis/c=48/22 les/c/f=49/27/0 sis=48) 2026-02-17T21:20:13.329 INFO:tasks.workunit.client.0.trial031.stdout:Import successful 2026-02-17T21:20:13.857 INFO:tasks.workunit.client.0.trial031.stdout:reviving divergent 1 2026-02-17T21:20:13.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:576: TEST_divergent_2: echo 'reviving divergent 1' 2026-02-17T21:20:13.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:577: TEST_divergent_2: ceph pg dump pgs 2026-02-17T21:20:14.047 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:20:14.047 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 100 0 0 70861 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:19:19.818123+0000 43'101 45:140 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:18:27.226024+0000 0'0 2026-02-17T21:18:27.226024+0000 0 0 periodic scrub scheduled @ 2026-02-19T05:02:25.367558+0000 0 0 2026-02-17T21:20:14.047 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:20:14.048 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:20:14.048 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:20:14.060 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:578: TEST_divergent_2: activate_osd td/divergent-priors 1 2026-02-17T21:20:14.060 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:20:14.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:20:14.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T21:20:14.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:20:14.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-17T21:20:14.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:20:14.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:20:14.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:20:14.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:20:14.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-17T21:20:14.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-17T21:20:14.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:20:14.062 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:20:14.062 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:20:14.062 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:20:14.062 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:20:14.062 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:20:14.062 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:20:14.062 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:20:14.062 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:20:14.062 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:20:14.062 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:20:14.062 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:20:14.062 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:20:14.062 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:20:14.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:20:14.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:20:14.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:20:14.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:20:14.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:20:14.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:20:14.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-17T21:20:14.064 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:20:14.065 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T21:20:14.065 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:20:14.065 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-17T21:20:14.065 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T21:20:14.065 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:20:14.065 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:20:14.065 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:20:14.082 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:14.079+0000 7f986c6848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:14.084 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:14.083+0000 7f986c6848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:14.085 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:14.083+0000 7f986c6848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:14.267 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T21:20:14.267 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:20:14.267 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:20:14.267 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:20:14.268 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:20:14.268 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:20:14.268 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:20:14.269 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:20:14.270 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:20:14.270 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:20:14.464 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 48 up_thru 48 down_at 30 last_clean_interval [9,28) [v2:127.0.0.1:6818/2358018890,v1:127.0.0.1:6819/2358018890] [v2:127.0.0.1:6820/2358018890,v1:127.0.0.1:6821/2358018890] exists,up d8cc5e2c-83e0-49d6-8a94-49a26ec2d2cb 2026-02-17T21:20:14.465 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:20:14.465 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:20:14.465 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:20:14.465 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:579: TEST_divergent_2: wait_for_osd up 1 2026-02-17T21:20:14.465 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:20:14.465 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:20:14.465 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:20:14.465 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:20:14.465 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:20:14.465 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:20:14.466 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:20:14.466 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:20:14.466 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:20:14.664 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 48 up_thru 48 down_at 30 last_clean_interval [9,28) [v2:127.0.0.1:6818/2358018890,v1:127.0.0.1:6819/2358018890] [v2:127.0.0.1:6820/2358018890,v1:127.0.0.1:6821/2358018890] exists,up d8cc5e2c-83e0-49d6-8a94-49a26ec2d2cb 2026-02-17T21:20:14.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:20:14.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:20:14.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:20:14.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:581: TEST_divergent_2: sleep 20 2026-02-17T21:20:15.417 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:15.415+0000 7f986c6848c0 -1 Falling back to public interface 2026-02-17T21:20:18.395 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:18.395+0000 7f986c6848c0 -1 osd.1 49 log_to_monitors true 2026-02-17T21:20:34.666 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:582: TEST_divergent_2: get_asok_path osd.1 2026-02-17T21:20:34.667 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T21:20:34.667 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T21:20:34.668 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:20:34.668 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:20:34.668 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:20:34.668 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.1.asok 2026-02-17T21:20:34.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:582: TEST_divergent_2: CEPH_ARGS= 2026-02-17T21:20:34.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:582: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.1.asok dump_ops_in_flight 2026-02-17T21:20:34.729 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:20:34.729 INFO:tasks.workunit.client.0.trial031.stdout: "ops": [], 2026-02-17T21:20:34.729 INFO:tasks.workunit.client.0.trial031.stdout: "num_ops": 0 2026-02-17T21:20:34.729 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:20:34.735 INFO:tasks.workunit.client.0.trial031.stdout:allowing recovery 2026-02-17T21:20:34.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:584: TEST_divergent_2: echo 'allowing recovery' 2026-02-17T21:20:34.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:585: TEST_divergent_2: ceph pg dump pgs 2026-02-17T21:20:34.933 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:20:34.934 INFO:tasks.workunit.client.0.trial031.stdout:1.0 101 0 0 0 0 71567 0 0 12 90 12 active+clean 2026-02-17T21:20:18.805667+0000 49'102 53:158 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:18:27.226024+0000 0'0 2026-02-17T21:18:27.226024+0000 0 0 periodic scrub scheduled @ 2026-02-19T08:22:05.998556+0000 0 0 2026-02-17T21:20:34.934 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:20:34.934 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:20:34.934 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:20:34.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:587: TEST_divergent_2: for i in $osds 2026-02-17T21:20:34.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:589: TEST_divergent_2: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-17T21:20:35.028 INFO:tasks.workunit.client.0.trial031.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-17T21:20:35.035 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:587: TEST_divergent_2: for i in $osds 2026-02-17T21:20:35.035 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:589: TEST_divergent_2: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-17T21:20:35.113 INFO:tasks.workunit.client.0.trial031.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-17T21:20:35.120 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:587: TEST_divergent_2: for i in $osds 2026-02-17T21:20:35.120 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:589: TEST_divergent_2: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-17T21:20:35.197 INFO:tasks.workunit.client.0.trial031.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-17T21:20:35.204 INFO:tasks.workunit.client.0.trial031.stdout:reading divergent objects 2026-02-17T21:20:35.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:592: TEST_divergent_2: echo 'reading divergent objects' 2026-02-17T21:20:35.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:593: TEST_divergent_2: ceph pg dump pgs 2026-02-17T21:20:35.401 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:20:35.402 INFO:tasks.workunit.client.0.trial031.stdout:1.0 101 0 0 0 0 71567 0 0 12 90 12 active+clean 2026-02-17T21:20:18.805667+0000 49'102 53:158 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:18:27.226024+0000 0'0 2026-02-17T21:18:27.226024+0000 0 0 periodic scrub scheduled @ 2026-02-19T08:22:05.998556+0000 0 0 2026-02-17T21:20:35.402 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:20:35.402 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:20:35.403 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:20:35.410 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: expr 2 + 2 2026-02-17T21:20:35.411 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: seq 1 4 2026-02-17T21:20:35.413 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:20:35.413 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_1 td/divergent-priors/existing 2026-02-17T21:20:35.433 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:20:35.433 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_2 td/divergent-priors/existing 2026-02-17T21:20:35.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:20:35.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_3 td/divergent-priors/existing 2026-02-17T21:20:35.487 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:20:35.487 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_4 td/divergent-priors/existing 2026-02-17T21:20:35.515 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:598: TEST_divergent_2: seq 1 2 2026-02-17T21:20:35.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:598: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-17T21:20:35.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:600: TEST_divergent_2: rados -p test get newobject_1 td/divergent-priors/existing 2026-02-17T21:20:35.536 INFO:tasks.workunit.client.0.trial031.stderr:error getting test/newobject_1: (2) No such file or directory 2026-02-17T21:20:35.537 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:598: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-17T21:20:35.537 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:600: TEST_divergent_2: rados -p test get newobject_2 td/divergent-priors/existing 2026-02-17T21:20:35.563 INFO:tasks.workunit.client.0.trial031.stderr:error getting test/newobject_2: (2) No such file or directory 2026-02-17T21:20:35.565 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:602: TEST_divergent_2: rm -f td/divergent-priors/existing 2026-02-17T21:20:35.566 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:604: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2026-02-17T21:20:35.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:604: TEST_divergent_2: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-17T21:20:35.571 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 20 _merge_object_divergent_entries: merging hoid 1:70b3bf12:::existing_4:head entries: 28'106 (28'4) delete 1:70b3bf12:::existing_4:head by client.4567.0:1 2026-02-17T21:18:40.960194+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: false, new_object: false 2026-02-17T21:20:35.571 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 20 _merge_object_divergent_entries: keeping 28'106 (28'4) delete 1:70b3bf12:::existing_4:head by client.4567.0:1 2026-02-17T21:18:40.960194+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: false, new_object: false 2026-02-17T21:20:35.571 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-17T21:20:35.571 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:70b3bf12:::existing_4:head prior_version: 28'4 first_divergent_update: 28'106 last_divergent_update: 28'106 2026-02-17T21:20:35.571 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: more recent entry found: 43'101 (28'4) modify 1:70b3bf12:::existing_4:head by client.4690.0:1 2026-02-17T21:19:34.445749+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false, already merged 2026-02-17T21:20:35.571 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 20 _merge_object_divergent_entries: merging hoid 1:75ac5091:::newobject_2:head entries: 28'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4554.0:1 2026-02-17T21:18:40.953252+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-17T21:20:35.571 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 20 _merge_object_divergent_entries: keeping 28'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4554.0:1 2026-02-17T21:18:40.953252+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-17T21:20:35.571 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-17T21:20:35.571 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version: 0'0 first_divergent_update: 28'101 last_divergent_update: 28'101 2026-02-17T21:20:35.571 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head has no more recent entries in log 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version or op type indicates creation, deleting 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 20 _merge_object_divergent_entries: merging hoid 1:8881e6d8:::existing_3:head entries: 28'105 (28'3) delete 1:8881e6d8:::existing_3:head by client.4566.0:1 2026-02-17T21:18:40.959766+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: false, new_object: false 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 20 _merge_object_divergent_entries: keeping 28'105 (28'3) delete 1:8881e6d8:::existing_3:head by client.4566.0:1 2026-02-17T21:18:40.959766+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: false, new_object: false 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid object_not_in_store: 1 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version: 28'3 first_divergent_update: 28'105 last_divergent_update: 28'105 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head has no more recent entries in log 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head must be rolled back or recovered, attempting to rollback 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head olog_can_rollback_to: 28'106 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot rollback 28'105 (28'3) delete 1:8881e6d8:::existing_3:head by client.4566.0:1 2026-02-17T21:18:40.959766+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: false, new_object: false 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot roll back, removing and adding to missing 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version 28'3 <= info.log_tail 28'90 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 20 _merge_object_divergent_entries: merging hoid 1:97db2a0f:::existing_2:head entries: 28'104 (28'2) modify 1:97db2a0f:::existing_2:head by client.4561.0:1 2026-02-17T21:18:40.957562+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 20 _merge_object_divergent_entries: keeping 28'104 (28'2) modify 1:97db2a0f:::existing_2:head by client.4561.0:1 2026-02-17T21:18:40.957562+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version: 28'2 first_divergent_update: 28'104 last_divergent_update: 28'104 2026-02-17T21:20:35.572 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head has no more recent entries in log 2026-02-17T21:20:35.573 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head must be rolled back or recovered, attempting to rollback 2026-02-17T21:20:35.573 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head olog_can_rollback_to: 28'106 2026-02-17T21:20:35.573 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot rollback 28'104 (28'2) modify 1:97db2a0f:::existing_2:head by client.4561.0:1 2026-02-17T21:18:40.957562+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:20:35.573 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot roll back, removing and adding to missing 2026-02-17T21:20:35.573 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version 28'2 <= info.log_tail 28'90 2026-02-17T21:20:35.573 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 20 _merge_object_divergent_entries: merging hoid 1:d858c035:::existing_1:head entries: 28'103 (28'1) modify 1:d858c035:::existing_1:head by client.4560.0:1 2026-02-17T21:18:40.956903+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:20:35.573 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 20 _merge_object_divergent_entries: keeping 28'103 (28'1) modify 1:d858c035:::existing_1:head by client.4560.0:1 2026-02-17T21:18:40.956903+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:20:35.573 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-17T21:20:35.573 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version: 28'1 first_divergent_update: 28'103 last_divergent_update: 28'103 2026-02-17T21:20:35.574 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head has no more recent entries in log 2026-02-17T21:20:35.574 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head must be rolled back or recovered, attempting to rollback 2026-02-17T21:20:35.574 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head olog_can_rollback_to: 28'106 2026-02-17T21:20:35.574 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot rollback 28'103 (28'1) modify 1:d858c035:::existing_1:head by client.4560.0:1 2026-02-17T21:18:40.956903+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:20:35.574 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot roll back, removing and adding to missing 2026-02-17T21:20:35.574 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version 28'1 <= info.log_tail 28'90 2026-02-17T21:20:35.574 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 20 _merge_object_divergent_entries: merging hoid 1:da59f6e7:::newobject_1:head entries: 28'102 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4555.0:1 2026-02-17T21:18:40.954512+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-17T21:20:35.574 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 20 _merge_object_divergent_entries: keeping 28'102 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4555.0:1 2026-02-17T21:18:40.954512+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-17T21:20:35.574 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-17T21:20:35.574 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version: 0'0 first_divergent_update: 28'102 last_divergent_update: 28'102 2026-02-17T21:20:35.574 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head has no more recent entries in log 2026-02-17T21:20:35.574 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:20:10.775+0000 7fcda60e9640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version or op type indicates creation, deleting 2026-02-17T21:20:35.575 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:606: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2026-02-17T21:20:35.575 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:606: TEST_divergent_2: grep -q '_merge_object_divergent_entries: more recent entry found:' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-17T21:20:35.577 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:612: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2026-02-17T21:20:35.578 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:612: TEST_divergent_2: grep -q '_merge_object_divergent_entries.*prior_version or op type indicates creation' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-17T21:20:35.580 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:617: TEST_divergent_2: echo success 2026-02-17T21:20:35.581 INFO:tasks.workunit.client.0.trial031.stdout:success 2026-02-17T21:20:35.581 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:619: TEST_divergent_2: rm td/divergent-priors/td/divergent-priors/exp.16076.out 2026-02-17T21:20:35.581 INFO:tasks.workunit.client.0.trial031.stderr:rm: cannot remove 'td/divergent-priors/td/divergent-priors/exp.16076.out': No such file or directory 2026-02-17T21:20:35.581 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:621: TEST_divergent_2: delete_pool test 2026-02-17T21:20:35.582 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=test 2026-02-17T21:20:35.582 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-17T21:20:35.823 INFO:tasks.workunit.client.0.trial031.stderr:pool 'test' does not exist 2026-02-17T21:20:35.832 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:622: TEST_divergent_2: kill_daemons td/divergent-priors 2026-02-17T21:20:35.833 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:20:35.834 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:20:35.834 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:20:35.834 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:20:35.834 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:20:44.149 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:20:44.149 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-17T21:20:44.149 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-17T21:20:44.149 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:20:44.149 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-17T21:20:44.150 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:20:44.150 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:20:44.150 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:20:44.150 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:20:44.150 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:20:44.158 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:20:44.158 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:20:44.159 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:20:44.160 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:20:44.160 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:20:44.161 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:20:44.161 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:20:44.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:20:44.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:20:44.163 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:20:44.163 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:20:44.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:20:44.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:20:44.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-17T21:20:44.186 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:20:44.186 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:20:44.186 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:20:44.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.16076 2026-02-17T21:20:44.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:20:44.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:20:44.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-17T21:20:44.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-17T21:20:44.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2026-02-17T21:20:44.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2026-02-17T21:20:44.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-17T21:20:44.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:20:44.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-17T21:20:44.188 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:20:44.188 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:20:44.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:20:44.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:20:44.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:20:44.191 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:20:44.191 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:20:44.192 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:20:44.192 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:20:44.193 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:20:44.193 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:20:44.193 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:20:44.195 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:20:44.195 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:20:44.195 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:20:44.195 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:20:44.196 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:20:44.197 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:20:44.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-17T21:20:44.199 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:20:44.199 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:20:44.199 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:20:44.199 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.16076 2026-02-17T21:20:44.200 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:20:44.200 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:20:44.200 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2026-02-17T21:20:44.202 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T21:20:44.202 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:20:44.202 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:20:44.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.16076 2026-02-17T21:20:44.203 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T21:20:44.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T21:20:44.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T21:20:44.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-17T21:20:44.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_3 td/divergent-priors 2026-02-17T21:20:44.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:628: TEST_divergent_3: local dir=td/divergent-priors 2026-02-17T21:20:44.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:631: TEST_divergent_3: local dummyfile=/etc/fstab 2026-02-17T21:20:44.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:632: TEST_divergent_3: local dummyfile2=/etc/resolv.conf 2026-02-17T21:20:44.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:634: TEST_divergent_3: local num_osds=3 2026-02-17T21:20:44.205 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:635: TEST_divergent_3: expr 3 - 1 2026-02-17T21:20:44.206 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:635: TEST_divergent_3: seq 0 2 2026-02-17T21:20:44.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:635: TEST_divergent_3: local 'osds=0 2026-02-17T21:20:44.207 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:20:44.207 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:20:44.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:636: TEST_divergent_3: run_mon td/divergent-priors a 2026-02-17T21:20:44.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2026-02-17T21:20:44.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T21:20:44.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T21:20:44.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T21:20:44.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2026-02-17T21:20:44.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-17T21:20:44.230 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T21:20:44.230 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:20:44.230 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:20:44.230 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:20:44.230 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:20:44.230 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:20:44.230 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:20:44.230 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T21:20:44.261 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T21:20:44.261 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T21:20:44.261 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:20:44.261 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:20:44.262 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T21:20:44.262 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T21:20:44.262 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:20:44.262 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:20:44.262 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:20:44.263 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:20:44.263 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:20:44.263 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:20:44.263 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-mon.a.asok 2026-02-17T21:20:44.263 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:20:44.263 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.16076/ceph-mon.a.asok config get fsid 2026-02-17T21:20:44.318 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T21:20:44.318 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:20:44.318 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:20:44.318 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T21:20:44.319 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T21:20:44.319 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:20:44.319 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:20:44.319 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:20:44.319 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:20:44.319 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:20:44.319 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:20:44.320 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-mon.a.asok 2026-02-17T21:20:44.320 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:20:44.320 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.16076/ceph-mon.a.asok config get mon_host 2026-02-17T21:20:44.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:637: TEST_divergent_3: run_mgr td/divergent-priors x 2026-02-17T21:20:44.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/divergent-priors 2026-02-17T21:20:44.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T21:20:44.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T21:20:44.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T21:20:44.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/divergent-priors/x 2026-02-17T21:20:44.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T21:20:44.490 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T21:20:44.490 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:20:44.490 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:20:44.490 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:20:44.490 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:20:44.490 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:20:44.491 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:20:44.491 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:20:44.492 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:638: TEST_divergent_3: for i in $osds 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: run_osd td/divergent-priors 0 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/0 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:20:44.512 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:20:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/0 2026-02-17T21:20:44.514 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:20:44.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=c804922e-7eb2-4582-9dc2-d361d3e2c25a 2026-02-17T21:20:44.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 c804922e-7eb2-4582-9dc2-d361d3e2c25a' 2026-02-17T21:20:44.515 INFO:tasks.workunit.client.0.trial031.stdout:add osd0 c804922e-7eb2-4582-9dc2-d361d3e2c25a 2026-02-17T21:20:44.516 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:20:44.529 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCs25RplmKZHxAAiFyS5NKIPl0yFzd/7pHw1A== 2026-02-17T21:20:44.529 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCs25RplmKZHxAAiFyS5NKIPl0yFzd/7pHw1A=="}' 2026-02-17T21:20:44.529 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new c804922e-7eb2-4582-9dc2-d361d3e2c25a -i td/divergent-priors/0/new.json 2026-02-17T21:20:44.641 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:20:44.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/0/new.json 2026-02-17T21:20:44.650 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCs25RplmKZHxAAiFyS5NKIPl0yFzd/7pHw1A== --osd-uuid c804922e-7eb2-4582-9dc2-d361d3e2c25a 2026-02-17T21:20:44.667 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:44.666+0000 7f1081c208c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:44.668 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:44.666+0000 7f1081c208c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:44.669 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:44.666+0000 7f1081c208c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:44.670 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:44.666+0000 7f1081c208c0 -1 bdev(0x563e6cd9fc00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-17T21:20:44.670 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:44.666+0000 7f1081c208c0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-17T21:20:47.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-17T21:20:47.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:20:47.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T21:20:47.002 INFO:tasks.workunit.client.0.trial031.stdout:adding osd0 key to auth repository 2026-02-17T21:20:47.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:20:47.282 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:20:47.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T21:20:47.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:20:47.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:20:47.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:20:47.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:20:47.310 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:47.306+0000 7f0840af38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:47.312 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:47.310+0000 7f0840af38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:47.313 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:47.310+0000 7f0840af38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:47.479 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T21:20:47.479 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:20:47.479 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:20:47.479 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:20:47.479 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:20:47.479 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:20:47.479 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:20:47.479 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:20:47.479 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:20:47.479 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:20:47.684 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:20:48.686 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:20:48.686 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:20:48.686 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:20:48.686 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:20:48.686 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:20:48.687 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:20:48.894 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:20:49.033 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:49.030+0000 7f0840af38c0 -1 Falling back to public interface 2026-02-17T21:20:49.895 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:20:49.895 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:20:49.895 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:20:49.896 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:20:49.896 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:20:49.896 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:20:50.102 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:20:51.104 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:20:51.104 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:20:51.104 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:20:51.104 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:20:51.104 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:20:51.104 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:20:51.307 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:20:52.038 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:52.033+0000 7f0840af38c0 -1 osd.0 0 log_to_monitors true 2026-02-17T21:20:52.308 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:20:52.308 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:20:52.308 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:20:52.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:20:52.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:20:52.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:20:52.511 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:20:53.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:20:53.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:20:53.513 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:20:53.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:20:53.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:20:53.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:20:53.715 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1013624114,v1:127.0.0.1:6803/1013624114] [v2:127.0.0.1:6804/1013624114,v1:127.0.0.1:6805/1013624114] exists,up c804922e-7eb2-4582-9dc2-d361d3e2c25a 2026-02-17T21:20:53.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:20:53.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:20:53.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:20:53.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:638: TEST_divergent_3: for i in $osds 2026-02-17T21:20:53.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: run_osd td/divergent-priors 1 2026-02-17T21:20:53.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-17T21:20:53.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:20:53.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T21:20:53.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:20:53.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/1 2026-02-17T21:20:53.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:20:53.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:20:53.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:20:53.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:20:53.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:20:53.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:20:53.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:20:53.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:20:53.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/1 2026-02-17T21:20:53.720 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:20:53.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2bdda522-e523-463c-b285-abb08be7107e 2026-02-17T21:20:53.720 INFO:tasks.workunit.client.0.trial031.stdout:add osd1 2bdda522-e523-463c-b285-abb08be7107e 2026-02-17T21:20:53.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 2bdda522-e523-463c-b285-abb08be7107e' 2026-02-17T21:20:53.721 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:20:53.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQC125RpNH3RKxAA6dsuPftmL899mmjMzineYA== 2026-02-17T21:20:53.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQC125RpNH3RKxAA6dsuPftmL899mmjMzineYA=="}' 2026-02-17T21:20:53.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2bdda522-e523-463c-b285-abb08be7107e -i td/divergent-priors/1/new.json 2026-02-17T21:20:53.935 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:20:53.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/1/new.json 2026-02-17T21:20:53.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC125RpNH3RKxAA6dsuPftmL899mmjMzineYA== --osd-uuid 2bdda522-e523-463c-b285-abb08be7107e 2026-02-17T21:20:53.963 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:53.961+0000 7f987c3f18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:53.964 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:53.961+0000 7f987c3f18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:53.965 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:53.961+0000 7f987c3f18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:53.965 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:53.961+0000 7f987c3f18c0 -1 bdev(0x561b9e5afc00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-17T21:20:53.966 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:53.961+0000 7f987c3f18c0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-17T21:20:56.216 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-17T21:20:56.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:20:56.217 INFO:tasks.workunit.client.0.trial031.stdout:adding osd1 key to auth repository 2026-02-17T21:20:56.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T21:20:56.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:20:56.497 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:20:56.498 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T21:20:56.498 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:20:56.498 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:20:56.499 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:20:56.499 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:20:56.515 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:56.513+0000 7f6fbcde98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:56.516 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:56.513+0000 7f6fbcde98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:56.517 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:56.513+0000 7f6fbcde98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:20:56.700 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T21:20:56.700 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:20:56.700 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:20:56.701 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:20:56.701 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:20:56.701 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:20:56.701 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:20:56.702 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:20:56.702 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:20:56.702 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:20:56.904 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:20:57.906 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:20:57.906 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:20:57.907 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:20:57.907 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:20:57.907 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:20:57.908 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:20:58.105 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:20:58.265 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:20:58.261+0000 7f6fbcde98c0 -1 Falling back to public interface 2026-02-17T21:20:59.106 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:20:59.106 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:20:59.107 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:20:59.107 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:20:59.107 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:20:59.107 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:20:59.306 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:00.307 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:00.308 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:00.308 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:21:00.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:21:00.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:00.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:21:00.503 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:01.302 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:01.297+0000 7f6fbcde98c0 -1 osd.1 0 log_to_monitors true 2026-02-17T21:21:01.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:01.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:01.505 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:21:01.506 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:21:01.506 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:01.506 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:21:01.709 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:02.711 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:02.711 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:02.711 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:21:02.712 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:21:02.712 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:02.712 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:21:02.910 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/376030295,v1:127.0.0.1:6811/376030295] [v2:127.0.0.1:6812/376030295,v1:127.0.0.1:6813/376030295] exists,up 2bdda522-e523-463c-b285-abb08be7107e 2026-02-17T21:21:02.914 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:21:02.914 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:21:02.915 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:21:02.915 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:638: TEST_divergent_3: for i in $osds 2026-02-17T21:21:02.915 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: run_osd td/divergent-priors 2 2026-02-17T21:21:02.915 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-17T21:21:02.915 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:21:02.915 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T21:21:02.915 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:21:02.915 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/2 2026-02-17T21:21:02.915 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:21:02.915 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:21:02.916 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:21:02.916 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:21:02.916 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-17T21:21:02.916 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-17T21:21:02.916 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:21:02.916 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:21:02.916 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:21:02.916 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:21:02.916 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:21:02.916 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:21:02.916 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:21:02.917 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:21:02.917 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:21:02.917 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:21:02.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:21:02.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:21:02.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:21:02.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:21:02.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:21:02.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:21:02.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:21:02.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:21:02.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:21:02.918 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:21:02.918 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:21:02.918 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:21:02.918 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/2 2026-02-17T21:21:02.918 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:21:02.918 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=258ca9b7-06c7-4314-ac32-bb7e4d29e018 2026-02-17T21:21:02.918 INFO:tasks.workunit.client.0.trial031.stdout:add osd2 258ca9b7-06c7-4314-ac32-bb7e4d29e018 2026-02-17T21:21:02.919 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 258ca9b7-06c7-4314-ac32-bb7e4d29e018' 2026-02-17T21:21:02.919 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:21:02.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQC+25RpM9BZNxAA/QT/z6oXqy7j6ipymUerqw== 2026-02-17T21:21:02.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQC+25RpM9BZNxAA/QT/z6oXqy7j6ipymUerqw=="}' 2026-02-17T21:21:02.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 258ca9b7-06c7-4314-ac32-bb7e4d29e018 -i td/divergent-priors/2/new.json 2026-02-17T21:21:03.125 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:21:03.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/2/new.json 2026-02-17T21:21:03.135 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC+25RpM9BZNxAA/QT/z6oXqy7j6ipymUerqw== --osd-uuid 258ca9b7-06c7-4314-ac32-bb7e4d29e018 2026-02-17T21:21:03.154 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:03.149+0000 7fda775068c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:21:03.156 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:03.153+0000 7fda775068c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:21:03.157 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:03.153+0000 7fda775068c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:21:03.158 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:03.153+0000 7fda775068c0 -1 bdev(0x55c67a8d3c00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-17T21:21:03.158 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:03.153+0000 7fda775068c0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-17T21:21:05.660 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-17T21:21:05.660 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:21:05.661 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T21:21:05.661 INFO:tasks.workunit.client.0.trial031.stdout:adding osd2 key to auth repository 2026-02-17T21:21:05.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:21:05.930 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T21:21:05.930 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:21:05.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:21:05.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:21:05.932 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:21:05.932 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:21:05.958 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:05.953+0000 7f9a636a48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:21:05.960 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:05.957+0000 7f9a636a48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:21:05.961 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:05.957+0000 7f9a636a48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:21:06.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T21:21:06.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:21:06.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:21:06.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:21:06.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:21:06.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:06.135 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:21:06.135 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:21:06.136 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:06.136 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:21:06.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:07.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:07.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:07.335 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:21:07.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:21:07.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:07.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:21:07.469 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:07.465+0000 7f9a636a48c0 -1 Falling back to public interface 2026-02-17T21:21:07.534 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:08.535 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:08.535 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:08.535 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:21:08.536 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:21:08.536 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:08.536 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:21:08.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:09.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:09.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:09.735 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:21:09.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:21:09.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:09.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:21:09.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:10.711 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:10.705+0000 7f9a636a48c0 -1 osd.2 0 log_to_monitors true 2026-02-17T21:21:10.930 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:10.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:10.931 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:21:10.932 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:21:10.932 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:10.932 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:21:11.133 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:12.135 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:12.135 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:12.135 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:21:12.136 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:21:12.136 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:12.136 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:21:12.335 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3807077142,v1:127.0.0.1:6819/3807077142] [v2:127.0.0.1:6820/3807077142,v1:127.0.0.1:6821/3807077142] exists,up 258ca9b7-06c7-4314-ac32-bb7e4d29e018 2026-02-17T21:21:12.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:21:12.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:21:12.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:21:12.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:643: TEST_divergent_3: ceph osd set noout 2026-02-17T21:21:12.636 INFO:tasks.workunit.client.0.trial031.stderr:noout is set 2026-02-17T21:21:12.647 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:644: TEST_divergent_3: ceph osd set noin 2026-02-17T21:21:12.941 INFO:tasks.workunit.client.0.trial031.stderr:noin is set 2026-02-17T21:21:12.954 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:645: TEST_divergent_3: ceph osd set nodown 2026-02-17T21:21:13.245 INFO:tasks.workunit.client.0.trial031.stderr:nodown is set 2026-02-17T21:21:13.254 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:646: TEST_divergent_3: create_pool test 1 1 2026-02-17T21:21:13.254 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T21:21:13.493 INFO:tasks.workunit.client.0.trial031.stderr:pool 'test' already exists 2026-02-17T21:21:13.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T21:21:14.503 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:647: TEST_divergent_3: ceph osd pool set test size 3 2026-02-17T21:21:14.806 INFO:tasks.workunit.client.0.trial031.stderr:set pool 1 size to 3 2026-02-17T21:21:14.817 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:648: TEST_divergent_3: ceph osd pool set test min_size 2 2026-02-17T21:21:15.111 INFO:tasks.workunit.client.0.trial031.stderr:set pool 1 min_size to 2 2026-02-17T21:21:15.121 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:651: TEST_divergent_3: ceph osd pool set test pg_autoscale_mode on 2026-02-17T21:21:15.417 INFO:tasks.workunit.client.0.trial031.stderr:set pool 1 pg_autoscale_mode to on 2026-02-17T21:21:15.429 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:653: TEST_divergent_3: divergent=-1 2026-02-17T21:21:15.430 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:654: TEST_divergent_3: date +%s 2026-02-17T21:21:15.431 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:654: TEST_divergent_3: start_time=1771363275 2026-02-17T21:21:15.431 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:655: TEST_divergent_3: max_duration=300 2026-02-17T21:21:15.431 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:657: TEST_divergent_3: '[' -1 -le -1 ']' 2026-02-17T21:21:15.431 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:659: TEST_divergent_3: flush_pg_stats 2026-02-17T21:21:15.431 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:21:15.432 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:21:15.637 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:21:15.637 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:21:15.637 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:21:15.637 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:21:15.637 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:21:15.638 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:21:15.716 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T21:21:15.716 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T21:21:15.716 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T21:21:15.716 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:21:15.716 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:21:15.793 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-17T21:21:15.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-17T21:21:15.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669' 2026-02-17T21:21:15.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:21:15.794 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:21:15.872 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574851 2026-02-17T21:21:15.872 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574851 2026-02-17T21:21:15.872 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-55834574851' 2026-02-17T21:21:15.872 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:21:15.873 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T21:21:15.873 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:21:15.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:21:15.874 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T21:21:15.874 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:21:15.876 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T21:21:15.876 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T21:21:15.876 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 21474836487 2026-02-17T21:21:15.877 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:21:16.080 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-17T21:21:16.081 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:21:17.081 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:21:17.082 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:21:17.281 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T21:21:17.282 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:21:17.282 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-17T21:21:17.282 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:21:17.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:21:17.284 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-17T21:21:17.284 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:21:17.285 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-17T21:21:17.286 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-17T21:21:17.286 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 38654705669 2026-02-17T21:21:17.286 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:21:17.499 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-17T21:21:17.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:21:17.500 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574851 2026-02-17T21:21:17.500 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:21:17.501 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:21:17.502 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574851 2026-02-17T21:21:17.502 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:21:17.504 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574851 2026-02-17T21:21:17.504 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-17T21:21:17.504 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 55834574851 2026-02-17T21:21:17.504 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:21:17.701 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-17T21:21:17.701 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:660: TEST_divergent_3: wait_for_clean 2026-02-17T21:21:17.701 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:21:17.702 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:21:17.702 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:21:17.702 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T21:21:17.703 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:21:17.703 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:21:17.703 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:21:17.703 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:21:17.703 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:21:17.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T21:21:17.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:21:17.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:21:17.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:21:17.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:21:17.812 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:21:18.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:21:18.016 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:21:18.016 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:21:18.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:21:18.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:21:18.017 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:21:18.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-17T21:21:18.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-17T21:21:18.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T21:21:18.100 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:21:18.100 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:21:18.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705672 2026-02-17T21:21:18.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705672 2026-02-17T21:21:18.177 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705672' 2026-02-17T21:21:18.177 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:21:18.177 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:21:18.256 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574854 2026-02-17T21:21:18.256 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574854 2026-02-17T21:21:18.256 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705672 2-55834574854' 2026-02-17T21:21:18.257 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:21:18.257 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-17T21:21:18.257 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:21:18.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:21:18.259 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-17T21:21:18.259 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:21:18.261 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-17T21:21:18.261 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T21:21:18.261 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 21474836489 2026-02-17T21:21:18.261 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:21:18.468 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-17T21:21:18.468 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:21:19.469 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:21:19.470 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:21:19.679 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836489 2026-02-17T21:21:19.679 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:21:19.679 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705672 2026-02-17T21:21:19.679 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:21:19.681 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:21:19.681 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705672 2026-02-17T21:21:19.682 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:21:19.683 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705672 2026-02-17T21:21:19.683 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705672' 2026-02-17T21:21:19.683 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 38654705672 2026-02-17T21:21:19.683 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:21:19.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705672 -lt 38654705672 2026-02-17T21:21:19.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:21:19.884 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574854 2026-02-17T21:21:19.884 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:21:19.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:21:19.886 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574854 2026-02-17T21:21:19.886 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:21:19.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574854 2026-02-17T21:21:19.888 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 55834574854 2026-02-17T21:21:19.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574854' 2026-02-17T21:21:19.888 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:21:20.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-17T21:21:20.099 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:21:20.100 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:21:20.100 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:21:20.366 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T21:21:20.367 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:21:20.367 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:21:20.367 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:21:20.368 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:21:20.368 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:21:20.368 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:21:20.368 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:21:20.563 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T21:21:20.564 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:21:20.564 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:21:20.564 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:21:20.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T21:21:20.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T21:21:20.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T21:21:20.822 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:663: TEST_divergent_3: ceph pg dump pgs --format=json 2026-02-17T21:21:20.822 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:663: TEST_divergent_3: jq '.pg_stats[0].up_primary' 2026-02-17T21:21:21.008 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:21:21.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:663: TEST_divergent_3: divergent=1 2026-02-17T21:21:21.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:664: TEST_divergent_3: echo 'primary and soon to be divergent is 1' 2026-02-17T21:21:21.019 INFO:tasks.workunit.client.0.trial031.stdout:primary and soon to be divergent is 1 2026-02-17T21:21:21.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:665: TEST_divergent_3: ceph pg dump pgs 2026-02-17T21:21:21.214 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:21:21.214 INFO:tasks.workunit.client.0.trial031.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T21:21:14.674588+0000 0'0 28:24 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T07:37:31.399446+0000 0 0 2026-02-17T21:21:21.214 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:21:21.214 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:21:21.215 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:21:21.222 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:667: TEST_divergent_3: date +%s 2026-02-17T21:21:21.223 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:667: TEST_divergent_3: current_time=1771363281 2026-02-17T21:21:21.223 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:668: TEST_divergent_3: expr 1771363281 - 1771363275 2026-02-17T21:21:21.225 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:668: TEST_divergent_3: elapsed_time=6 2026-02-17T21:21:21.226 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:669: TEST_divergent_3: '[' 6 -gt 300 ']' 2026-02-17T21:21:21.226 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:657: TEST_divergent_3: '[' 1 -le -1 ']' 2026-02-17T21:21:21.226 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:675: TEST_divergent_3: local non_divergent= 2026-02-17T21:21:21.226 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:676: TEST_divergent_3: for i in $osds 2026-02-17T21:21:21.226 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: '[' 0 = 1 ']' 2026-02-17T21:21:21.226 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:681: TEST_divergent_3: non_divergent=' 0' 2026-02-17T21:21:21.226 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:676: TEST_divergent_3: for i in $osds 2026-02-17T21:21:21.226 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: '[' 1 = 1 ']' 2026-02-17T21:21:21.226 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:679: TEST_divergent_3: continue 2026-02-17T21:21:21.226 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:676: TEST_divergent_3: for i in $osds 2026-02-17T21:21:21.226 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: '[' 2 = 1 ']' 2026-02-17T21:21:21.227 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:681: TEST_divergent_3: non_divergent=' 0 2' 2026-02-17T21:21:21.227 INFO:tasks.workunit.client.0.trial031.stdout:writing initial objects 2026-02-17T21:21:21.228 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:684: TEST_divergent_3: echo 'writing initial objects' 2026-02-17T21:21:21.228 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: seq 1 100 2026-02-17T21:21:21.228 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.228 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_1 /etc/fstab 2026-02-17T21:21:21.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_2 /etc/fstab 2026-02-17T21:21:21.275 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.275 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_3 /etc/fstab 2026-02-17T21:21:21.302 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.303 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_4 /etc/fstab 2026-02-17T21:21:21.330 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.330 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_5 /etc/fstab 2026-02-17T21:21:21.358 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.359 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_6 /etc/fstab 2026-02-17T21:21:21.386 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.386 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_7 /etc/fstab 2026-02-17T21:21:21.413 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.413 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_8 /etc/fstab 2026-02-17T21:21:21.441 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.441 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_9 /etc/fstab 2026-02-17T21:21:21.466 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.466 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_10 /etc/fstab 2026-02-17T21:21:21.494 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.494 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_11 /etc/fstab 2026-02-17T21:21:21.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_12 /etc/fstab 2026-02-17T21:21:21.548 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.549 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_13 /etc/fstab 2026-02-17T21:21:21.574 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.574 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_14 /etc/fstab 2026-02-17T21:21:21.604 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.604 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_15 /etc/fstab 2026-02-17T21:21:21.633 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.633 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_16 /etc/fstab 2026-02-17T21:21:21.660 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.661 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_17 /etc/fstab 2026-02-17T21:21:21.689 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.689 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_18 /etc/fstab 2026-02-17T21:21:21.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_19 /etc/fstab 2026-02-17T21:21:21.746 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.746 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_20 /etc/fstab 2026-02-17T21:21:21.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_21 /etc/fstab 2026-02-17T21:21:21.802 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.802 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_22 /etc/fstab 2026-02-17T21:21:21.830 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.830 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_23 /etc/fstab 2026-02-17T21:21:21.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_24 /etc/fstab 2026-02-17T21:21:21.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_25 /etc/fstab 2026-02-17T21:21:21.913 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.913 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_26 /etc/fstab 2026-02-17T21:21:21.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_27 /etc/fstab 2026-02-17T21:21:21.968 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.969 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_28 /etc/fstab 2026-02-17T21:21:21.997 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:21.997 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_29 /etc/fstab 2026-02-17T21:21:22.024 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.024 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_30 /etc/fstab 2026-02-17T21:21:22.052 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.052 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_31 /etc/fstab 2026-02-17T21:21:22.080 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.080 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_32 /etc/fstab 2026-02-17T21:21:22.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_33 /etc/fstab 2026-02-17T21:21:22.137 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.137 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_34 /etc/fstab 2026-02-17T21:21:22.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_35 /etc/fstab 2026-02-17T21:21:22.192 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.192 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_36 /etc/fstab 2026-02-17T21:21:22.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_37 /etc/fstab 2026-02-17T21:21:22.246 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.246 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_38 /etc/fstab 2026-02-17T21:21:22.274 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.274 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_39 /etc/fstab 2026-02-17T21:21:22.301 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.302 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_40 /etc/fstab 2026-02-17T21:21:22.329 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.329 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_41 /etc/fstab 2026-02-17T21:21:22.358 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.358 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_42 /etc/fstab 2026-02-17T21:21:22.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.386 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_43 /etc/fstab 2026-02-17T21:21:22.414 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.414 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_44 /etc/fstab 2026-02-17T21:21:22.441 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.441 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_45 /etc/fstab 2026-02-17T21:21:22.468 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.468 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_46 /etc/fstab 2026-02-17T21:21:22.496 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.496 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_47 /etc/fstab 2026-02-17T21:21:22.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_48 /etc/fstab 2026-02-17T21:21:22.552 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.552 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_49 /etc/fstab 2026-02-17T21:21:22.580 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.580 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_50 /etc/fstab 2026-02-17T21:21:22.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.609 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_51 /etc/fstab 2026-02-17T21:21:22.637 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.637 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_52 /etc/fstab 2026-02-17T21:21:22.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_53 /etc/fstab 2026-02-17T21:21:22.693 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.693 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_54 /etc/fstab 2026-02-17T21:21:22.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_55 /etc/fstab 2026-02-17T21:21:22.745 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.745 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_56 /etc/fstab 2026-02-17T21:21:22.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_57 /etc/fstab 2026-02-17T21:21:22.801 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.801 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_58 /etc/fstab 2026-02-17T21:21:22.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_59 /etc/fstab 2026-02-17T21:21:22.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_60 /etc/fstab 2026-02-17T21:21:22.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_61 /etc/fstab 2026-02-17T21:21:22.913 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.913 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_62 /etc/fstab 2026-02-17T21:21:22.940 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.940 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_63 /etc/fstab 2026-02-17T21:21:22.968 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.968 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_64 /etc/fstab 2026-02-17T21:21:22.996 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:22.996 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_65 /etc/fstab 2026-02-17T21:21:23.023 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.023 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_66 /etc/fstab 2026-02-17T21:21:23.043 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.043 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_67 /etc/fstab 2026-02-17T21:21:23.071 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_68 /etc/fstab 2026-02-17T21:21:23.092 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.092 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_69 /etc/fstab 2026-02-17T21:21:23.118 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.119 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_70 /etc/fstab 2026-02-17T21:21:23.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.146 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_71 /etc/fstab 2026-02-17T21:21:23.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_72 /etc/fstab 2026-02-17T21:21:23.200 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.200 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_73 /etc/fstab 2026-02-17T21:21:23.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_74 /etc/fstab 2026-02-17T21:21:23.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_75 /etc/fstab 2026-02-17T21:21:23.266 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.267 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_76 /etc/fstab 2026-02-17T21:21:23.294 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.294 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_77 /etc/fstab 2026-02-17T21:21:23.321 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.321 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_78 /etc/fstab 2026-02-17T21:21:23.349 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.349 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_79 /etc/fstab 2026-02-17T21:21:23.377 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.377 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_80 /etc/fstab 2026-02-17T21:21:23.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_81 /etc/fstab 2026-02-17T21:21:23.432 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.432 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_82 /etc/fstab 2026-02-17T21:21:23.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_83 /etc/fstab 2026-02-17T21:21:23.489 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.489 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_84 /etc/fstab 2026-02-17T21:21:23.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.517 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_85 /etc/fstab 2026-02-17T21:21:23.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_86 /etc/fstab 2026-02-17T21:21:23.571 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.571 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_87 /etc/fstab 2026-02-17T21:21:23.599 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.600 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_88 /etc/fstab 2026-02-17T21:21:23.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_89 /etc/fstab 2026-02-17T21:21:23.640 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.640 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_90 /etc/fstab 2026-02-17T21:21:23.666 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.666 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_91 /etc/fstab 2026-02-17T21:21:23.693 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.694 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_92 /etc/fstab 2026-02-17T21:21:23.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_93 /etc/fstab 2026-02-17T21:21:23.749 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.749 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_94 /etc/fstab 2026-02-17T21:21:23.776 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.777 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_95 /etc/fstab 2026-02-17T21:21:23.804 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.804 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_96 /etc/fstab 2026-02-17T21:21:23.833 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.833 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_97 /etc/fstab 2026-02-17T21:21:23.861 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.861 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_98 /etc/fstab 2026-02-17T21:21:23.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_99 /etc/fstab 2026-02-17T21:21:23.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-17T21:21:23.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_100 /etc/fstab 2026-02-17T21:21:23.945 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:691: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-17T21:21:23.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:691: TEST_divergent_3: wait_for_clean 2026-02-17T21:21:23.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:21:23.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:21:23.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:21:23.946 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-17T21:21:23.947 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:21:23.947 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:21:23.947 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:21:23.948 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:21:23.948 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:21:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-17T21:21:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:21:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:21:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:21:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:21:24.019 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:21:24.223 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:21:24.223 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:21:24.223 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:21:24.224 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:21:24.224 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:21:24.224 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:21:24.301 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836493 2026-02-17T21:21:24.302 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836493 2026-02-17T21:21:24.302 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493' 2026-02-17T21:21:24.302 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:21:24.302 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:21:24.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-17T21:21:24.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-17T21:21:24.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493 1-38654705675' 2026-02-17T21:21:24.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:21:24.385 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:21:24.466 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574857 2026-02-17T21:21:24.466 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574857 2026-02-17T21:21:24.466 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493 1-38654705675 2-55834574857' 2026-02-17T21:21:24.466 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:21:24.467 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836493 2026-02-17T21:21:24.467 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:21:24.468 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:21:24.469 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836493 2026-02-17T21:21:24.469 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:21:24.470 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836493 2026-02-17T21:21:24.470 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-17T21:21:24.471 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 21474836493 2026-02-17T21:21:24.471 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:21:24.683 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836493 2026-02-17T21:21:24.683 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:21:25.684 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:21:25.684 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:21:25.883 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836493 2026-02-17T21:21:25.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:21:25.889 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705675 2026-02-17T21:21:25.890 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:21:25.890 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:21:25.890 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-17T21:21:25.890 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:21:25.890 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-17T21:21:25.890 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-17T21:21:25.890 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 38654705675 2026-02-17T21:21:25.891 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:21:26.096 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-17T21:21:26.097 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:21:26.097 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574857 2026-02-17T21:21:26.098 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:21:26.098 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:21:26.099 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574857 2026-02-17T21:21:26.099 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:21:26.100 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574857 2026-02-17T21:21:26.101 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 55834574857 2026-02-17T21:21:26.101 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-17T21:21:26.101 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:21:26.305 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-17T21:21:26.306 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:21:26.306 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:21:26.306 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:21:26.566 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T21:21:26.566 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:21:26.566 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:21:26.566 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:21:26.566 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:21:26.566 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:21:26.566 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:21:26.566 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:21:26.759 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T21:21:26.760 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:21:26.760 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:21:26.760 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:21:27.022 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T21:21:27.022 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T21:21:27.022 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T21:21:27.022 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:693: TEST_divergent_3: get_pg test existing_1 2026-02-17T21:21:27.022 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-17T21:21:27.022 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=existing_1 2026-02-17T21:21:27.022 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test existing_1 2026-02-17T21:21:27.022 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T21:21:27.222 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:693: TEST_divergent_3: local pgid=1.0 2026-02-17T21:21:27.222 INFO:tasks.workunit.client.0.trial031.stdout:blackholing osds 0 2 2026-02-17T21:21:27.223 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:696: TEST_divergent_3: echo 'blackholing osds 0 2' 2026-02-17T21:21:27.223 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:697: TEST_divergent_3: ceph pg dump pgs 2026-02-17T21:21:27.415 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:21:27.415 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 0 0 0 70600 0 0 16 84 16 active+clean 2026-02-17T21:21:14.674588+0000 28'100 28:124 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T07:37:31.399446+0000 0 0 2026-02-17T21:21:27.415 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:21:27.415 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:21:27.415 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:21:27.423 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:698: TEST_divergent_3: for i in $non_divergent 2026-02-17T21:21:27.424 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: get_asok_path osd.0 2026-02-17T21:21:27.424 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T21:21:27.424 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T21:21:27.424 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:21:27.424 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:21:27.424 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:21:27.425 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.0.asok 2026-02-17T21:21:27.425 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: CEPH_ARGS= 2026-02-17T21:21:27.425 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-17T21:21:27.478 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:21:27.478 INFO:tasks.workunit.client.0.trial031.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-17T21:21:27.478 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:21:27.483 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:698: TEST_divergent_3: for i in $non_divergent 2026-02-17T21:21:27.484 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: get_asok_path osd.2 2026-02-17T21:21:27.484 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-17T21:21:27.484 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-17T21:21:27.485 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:21:27.485 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:21:27.485 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:21:27.485 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.2.asok 2026-02-17T21:21:27.485 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: CEPH_ARGS= 2026-02-17T21:21:27.486 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-17T21:21:27.537 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:21:27.537 INFO:tasks.workunit.client.0.trial031.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-17T21:21:27.537 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:21:27.543 INFO:tasks.workunit.client.0.trial031.stdout:create new divergent objects 2026-02-17T21:21:27.543 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:704: TEST_divergent_3: echo 'create new divergent objects' 2026-02-17T21:21:27.543 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:705: TEST_divergent_3: seq 1 2 2026-02-17T21:21:27.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:705: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-17T21:21:27.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:705: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-17T21:21:27.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:707: TEST_divergent_3: rados -p test create newobject_1 2026-02-17T21:21:27.545 INFO:tasks.workunit.client.0.trial031.stdout:writing divergent objects 2026-02-17T21:21:27.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:710: TEST_divergent_3: echo 'writing divergent objects' 2026-02-17T21:21:27.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:707: TEST_divergent_3: rados -p test create newobject_2 2026-02-17T21:21:27.546 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:711: TEST_divergent_3: seq 1 2 2026-02-17T21:21:27.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:711: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-17T21:21:27.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:711: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-17T21:21:27.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:713: TEST_divergent_3: rados -p test put existing_1 /etc/resolv.conf 2026-02-17T21:21:27.547 INFO:tasks.workunit.client.0.trial031.stdout:remove divergent objects 2026-02-17T21:21:27.547 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:716: TEST_divergent_3: echo 'remove divergent objects' 2026-02-17T21:21:27.547 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:713: TEST_divergent_3: rados -p test put existing_2 /etc/resolv.conf 2026-02-17T21:21:27.548 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:717: TEST_divergent_3: seq 1 2 2026-02-17T21:21:27.548 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:717: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-17T21:21:27.548 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: expr 1 + 2 2026-02-17T21:21:27.550 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: rmi=3 2026-02-17T21:21:27.550 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:717: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-17T21:21:27.550 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:720: TEST_divergent_3: rados -p test rm existing_3 2026-02-17T21:21:27.550 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: expr 2 + 2 2026-02-17T21:21:27.551 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: rmi=4 2026-02-17T21:21:27.552 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:722: TEST_divergent_3: sleep 10 2026-02-17T21:21:27.552 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:720: TEST_divergent_3: rados -p test rm existing_4 2026-02-17T21:21:37.554 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:723: TEST_divergent_3: killall -9 rados 2026-02-17T21:21:37.590 INFO:tasks.workunit.client.0.trial031.stderr:rados: no process found 2026-02-17T21:21:37.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:726: TEST_divergent_3: echo 'killing all the osds' 2026-02-17T21:21:37.591 INFO:tasks.workunit.client.0.trial031.stdout:killing all the osds 2026-02-17T21:21:37.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:727: TEST_divergent_3: ceph pg dump pgs 2026-02-17T21:21:37.794 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:21:37.795 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 0 0 0 69710 0 0 16 90 16 active+clean 2026-02-17T21:21:14.674588+0000 28'106 28:130 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T07:37:31.399446+0000 0 0 2026-02-17T21:21:37.795 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:21:37.795 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:21:37.795 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:21:37.803 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:728: TEST_divergent_3: kill_daemons td/divergent-priors KILL osd 2026-02-17T21:21:37.803 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:21:37.804 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:21:37.804 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:21:37.804 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:21:37.804 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:21:37.913 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:21:37.913 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:729: TEST_divergent_3: for i in $osds 2026-02-17T21:21:37.913 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:731: TEST_divergent_3: ceph osd down osd.0 2026-02-17T21:21:38.161 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already down. 2026-02-17T21:21:38.170 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:729: TEST_divergent_3: for i in $osds 2026-02-17T21:21:38.170 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:731: TEST_divergent_3: ceph osd down osd.1 2026-02-17T21:21:38.431 INFO:tasks.workunit.client.0.trial031.stderr:osd.1 is already down. 2026-02-17T21:21:38.440 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:729: TEST_divergent_3: for i in $osds 2026-02-17T21:21:38.440 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:731: TEST_divergent_3: ceph osd down osd.2 2026-02-17T21:21:38.737 INFO:tasks.workunit.client.0.trial031.stderr:osd.2 is already down. 2026-02-17T21:21:38.746 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:733: TEST_divergent_3: for i in $non_divergent 2026-02-17T21:21:38.747 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:735: TEST_divergent_3: ceph osd out osd.0 2026-02-17T21:21:39.009 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already out. 2026-02-17T21:21:39.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:733: TEST_divergent_3: for i in $non_divergent 2026-02-17T21:21:39.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:735: TEST_divergent_3: ceph osd out osd.2 2026-02-17T21:21:39.282 INFO:tasks.workunit.client.0.trial031.stderr:osd.2 is already out. 2026-02-17T21:21:39.292 INFO:tasks.workunit.client.0.trial031.stdout:bringing up non_divergent 0 2 2026-02-17T21:21:39.292 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:739: TEST_divergent_3: echo 'bringing up non_divergent 0 2' 2026-02-17T21:21:39.292 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:740: TEST_divergent_3: ceph pg dump pgs 2026-02-17T21:21:39.490 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:21:39.490 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 0 0 0 69710 0 0 16 90 16 stale+active+clean 2026-02-17T21:21:14.674588+0000 28'106 28:130 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T07:37:31.399446+0000 0 0 2026-02-17T21:21:39.490 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:21:39.490 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:21:39.490 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:21:39.499 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:741: TEST_divergent_3: for i in $non_divergent 2026-02-17T21:21:39.499 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:743: TEST_divergent_3: activate_osd td/divergent-priors 0 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/0 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:21:39.500 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:21:39.501 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:21:39.501 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:21:39.501 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:21:39.501 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:21:39.501 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:21:39.501 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:21:39.501 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:21:39.501 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:21:39.501 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:21:39.501 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:21:39.501 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:21:39.501 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/0 2026-02-17T21:21:39.502 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:21:39.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T21:21:39.503 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:21:39.503 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/0/whoami 2026-02-17T21:21:39.504 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T21:21:39.504 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:21:39.504 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:21:39.504 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:21:39.528 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:39.527+0000 7f28229f68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:21:39.529 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:39.527+0000 7f28229f68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:21:39.530 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:39.527+0000 7f28229f68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:21:39.711 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T21:21:39.711 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:21:39.711 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:21:39.711 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:21:39.711 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:21:39.711 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:39.711 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:21:39.711 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:21:39.711 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:39.712 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:21:39.916 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:40.805 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:40.803+0000 7f28229f68c0 -1 Falling back to public interface 2026-02-17T21:21:40.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:40.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:40.918 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:21:40.918 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:21:40.918 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:40.918 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:21:41.119 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:42.120 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:42.120 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:42.121 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:21:42.121 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:21:42.121 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:42.122 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:21:42.318 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:43.320 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:43.320 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:43.320 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:21:43.321 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:21:43.321 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:43.321 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:21:43.518 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:44.078 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:44.075+0000 7f28229f68c0 -1 osd.0 28 log_to_monitors true 2026-02-17T21:21:44.519 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:44.520 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:44.520 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:21:44.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:21:44.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:44.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:21:44.719 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up out weight 0 up_from 35 up_thru 0 down_at 29 last_clean_interval [5,28) [v2:127.0.0.1:6802/3194159082,v1:127.0.0.1:6803/3194159082] [v2:127.0.0.1:6804/3194159082,v1:127.0.0.1:6805/3194159082] exists,up c804922e-7eb2-4582-9dc2-d361d3e2c25a 2026-02-17T21:21:44.723 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:21:44.723 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:21:44.723 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:21:44.723 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:741: TEST_divergent_3: for i in $non_divergent 2026-02-17T21:21:44.723 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:743: TEST_divergent_3: activate_osd td/divergent-priors 2 2026-02-17T21:21:44.723 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:21:44.723 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:21:44.723 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T21:21:44.723 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:21:44.724 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/2 2026-02-17T21:21:44.724 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:21:44.724 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:21:44.724 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:21:44.724 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:21:44.724 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-17T21:21:44.724 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-17T21:21:44.724 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:21:44.724 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:21:44.724 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:21:44.724 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:21:44.724 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:21:44.725 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:21:44.725 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:21:44.725 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:21:44.725 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:21:44.725 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:21:44.725 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:21:44.725 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:21:44.725 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:21:44.725 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:21:44.725 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:21:44.725 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:21:44.726 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:21:44.726 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:21:44.726 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:21:44.726 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:21:44.726 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/2 2026-02-17T21:21:44.726 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:21:44.727 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T21:21:44.727 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:21:44.728 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/2/whoami 2026-02-17T21:21:44.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T21:21:44.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:21:44.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:21:44.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:21:44.743 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:44.739+0000 7f5611e538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:21:44.744 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:44.743+0000 7f5611e538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:21:44.745 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:44.743+0000 7f5611e538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:21:44.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T21:21:44.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:21:44.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:21:44.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:21:44.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:21:44.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:44.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:21:44.931 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:21:44.932 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:44.932 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:21:45.130 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:45.981 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:45.979+0000 7f5611e538c0 -1 Falling back to public interface 2026-02-17T21:21:46.131 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:46.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:46.132 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:21:46.133 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:21:46.133 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:46.133 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:21:46.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:47.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:47.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:47.334 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:21:47.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:21:47.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:47.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:21:47.533 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:48.534 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:48.534 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:48.534 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:21:48.535 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:21:48.535 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:48.535 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:21:48.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:21:49.233 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:21:49.231+0000 7f5611e538c0 -1 osd.2 28 log_to_monitors true 2026-02-17T21:21:49.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:21:49.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:21:49.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:21:49.736 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:21:49.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:21:49.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:21:49.933 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up out weight 0 up_from 40 up_thru 0 down_at 31 last_clean_interval [13,28) [v2:127.0.0.1:6810/1416133595,v1:127.0.0.1:6811/1416133595] [v2:127.0.0.1:6812/1416133595,v1:127.0.0.1:6813/1416133595] exists,up 258ca9b7-06c7-4314-ac32-bb7e4d29e018 2026-02-17T21:21:49.934 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:21:49.934 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:21:49.934 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:21:49.935 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:745: TEST_divergent_3: for i in $non_divergent 2026-02-17T21:21:49.935 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:747: TEST_divergent_3: ceph osd in osd.0 2026-02-17T21:21:50.186 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already in. 2026-02-17T21:21:50.197 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:745: TEST_divergent_3: for i in $non_divergent 2026-02-17T21:21:50.197 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:747: TEST_divergent_3: ceph osd in osd.2 2026-02-17T21:21:50.494 INFO:tasks.workunit.client.0.trial031.stderr:osd.2 is already in. 2026-02-17T21:21:50.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:750: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-17T21:21:50.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:750: TEST_divergent_3: wait_for_clean 2026-02-17T21:21:50.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:21:50.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:21:50.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:21:50.506 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-17T21:21:50.506 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:21:50.507 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:21:50.507 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:21:50.507 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:21:50.507 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:21:50.571 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-17T21:21:50.571 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:21:50.571 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:21:50.571 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:21:50.571 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:21:50.571 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:21:50.781 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:21:50.781 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:21:50.781 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:21:50.781 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:21:50.781 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:21:50.781 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:21:50.862 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855364 2026-02-17T21:21:50.863 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855364 2026-02-17T21:21:50.863 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855364' 2026-02-17T21:21:50.863 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:21:50.863 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:21:50.921 INFO:tasks.workunit.client.0.trial031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-17T21:21:50.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-17T21:21:50.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-17T21:21:50.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-17T21:21:50.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:21:50.925 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:21:51.004 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691843 2026-02-17T21:21:51.005 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691843 2026-02-17T21:21:51.005 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855364 2-171798691843' 2026-02-17T21:21:51.005 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:21:51.005 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855364 2026-02-17T21:21:51.005 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:21:51.006 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:21:51.007 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855364 2026-02-17T21:21:51.007 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:21:51.008 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855364 2026-02-17T21:21:51.008 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855364' 2026-02-17T21:21:51.009 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 150323855364 2026-02-17T21:21:51.009 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:21:51.214 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855362 -lt 150323855364 2026-02-17T21:21:51.214 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:21:52.215 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:21:52.216 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:21:52.420 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855362 -lt 150323855364 2026-02-17T21:21:52.420 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:21:53.421 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T21:21:53.422 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:21:53.623 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855364 -lt 150323855364 2026-02-17T21:21:53.623 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:21:53.624 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-171798691843 2026-02-17T21:21:53.624 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:21:53.625 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:21:53.626 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-171798691843 2026-02-17T21:21:53.626 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:21:53.627 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691843 2026-02-17T21:21:53.628 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 171798691843 2026-02-17T21:21:53.628 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 171798691843' 2026-02-17T21:21:53.628 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:21:53.845 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691843 -lt 171798691843 2026-02-17T21:21:53.845 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:21:53.846 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:21:53.846 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:21:54.116 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 == 0 2026-02-17T21:21:54.116 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:21:54.117 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:21:54.117 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:21:54.117 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:21:54.117 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:21:54.117 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:21:54.117 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:21:54.324 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:21:54.325 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:21:54.325 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:21:54.325 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:21:54.583 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:21:54.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T21:21:54.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T21:21:54.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T21:21:54.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:21:54.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T21:21:54.684 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:21:54.684 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:21:54.685 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:21:54.685 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:21:54.685 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:21:54.686 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:21:54.686 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:21:54.686 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:21:54.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:21:54.886 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:21:54.886 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:21:54.886 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:21:55.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:21:55.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:21:55.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:21:55.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:21:55.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:21:55.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:21:55.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:21:55.145 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:21:55.146 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:21:55.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:21:55.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:21:55.407 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-17T21:21:55.407 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:21:55.407 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T21:21:55.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:21:55.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:21:55.608 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:21:55.608 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:21:55.609 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:21:55.609 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:21:55.609 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:21:55.609 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:21:55.804 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:21:55.805 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:21:55.805 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:21:55.806 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:21:56.062 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:21:56.062 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:21:56.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:21:56.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:21:56.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:21:56.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:21:56.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:21:56.063 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:21:56.063 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:21:56.318 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:21:56.318 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:21:56.318 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-17T21:21:56.318 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:21:56.318 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T21:21:56.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:21:56.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:21:56.720 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:21:56.720 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:21:56.720 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:21:56.720 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:21:56.720 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:21:56.720 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:21:56.918 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:21:56.919 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:21:56.919 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:21:56.920 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:21:57.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:21:57.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:21:57.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:21:57.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:21:57.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:21:57.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:21:57.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:21:57.176 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:21:57.176 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:21:57.434 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:21:57.434 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:21:57.435 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-17T21:21:57.435 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:21:57.435 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T21:21:58.235 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:21:58.235 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:21:58.236 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:21:58.236 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:21:58.236 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:21:58.237 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:21:58.237 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:21:58.237 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:21:58.436 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:21:58.437 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:21:58.437 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:21:58.437 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:21:58.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:21:58.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:21:58.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:21:58.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:21:58.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:21:58.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:21:58.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:21:58.696 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:21:58.696 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:21:58.956 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:21:58.956 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:21:58.956 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-17T21:21:58.956 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:21:58.956 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-17T21:22:00.557 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:22:00.557 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:22:00.558 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:22:00.558 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:22:00.558 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:22:00.559 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:22:00.559 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:22:00.559 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:22:00.756 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:22:00.757 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:22:00.757 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:22:00.757 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:22:01.023 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:22:01.023 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:22:01.023 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:22:01.023 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:22:01.024 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:22:01.024 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:22:01.024 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:22:01.024 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:22:01.024 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:22:01.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:22:01.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:22:01.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-17T21:22:01.284 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:22:01.284 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-17T21:22:04.484 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:22:04.484 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:22:04.485 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:22:04.485 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:22:04.485 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:22:04.486 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:22:04.486 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:22:04.486 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:22:04.686 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:22:04.687 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:22:04.687 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:22:04.687 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:22:04.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:22:04.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:22:04.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:22:04.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:22:04.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:22:04.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:22:04.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:22:04.944 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:22:04.945 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:22:05.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:22:05.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:22:05.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-17T21:22:05.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:22:05.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-17T21:22:11.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:22:11.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:22:11.608 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:22:11.609 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:22:11.609 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:22:11.609 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:22:11.609 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:22:11.609 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:22:11.807 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:22:11.808 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:22:11.808 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:22:11.808 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:22:12.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:22:12.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:22:12.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:22:12.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:22:12.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:22:12.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:22:12.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:22:12.072 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:22:12.072 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:22:12.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:22:12.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:22:12.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-17T21:22:12.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:22:12.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-17T21:22:19.640 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:22:19.640 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:22:19.641 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:22:19.641 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:22:19.641 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:22:19.641 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:22:19.641 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:22:19.641 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:22:19.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:22:19.851 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:22:19.851 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:22:19.851 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:22:20.114 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:22:20.114 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:22:20.114 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:22:20.114 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:22:20.114 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:22:20.115 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:22:20.115 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:22:20.115 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:22:20.115 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:22:20.384 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:22:20.384 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:22:20.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-17T21:22:20.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-17T21:22:20.640 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:22:20.640 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_fingerprint": "f47ed877-51eb-480a-9d76-c2fb240ab7b0", 2026-02-17T21:22:20.641 INFO:tasks.workunit.client.0.trial031.stdout: "version": "19.2.3-679-gd855f53b", 2026-02-17T21:22:20.641 INFO:tasks.workunit.client.0.trial031.stdout: "commit": "d855f53b89fdcec760fd9232a5fb55ed4fb111a1", 2026-02-17T21:22:20.641 INFO:tasks.workunit.client.0.trial031.stdout: "timestamp": "2026-02-17T21:22:20.639224+0000", 2026-02-17T21:22:20.641 INFO:tasks.workunit.client.0.trial031.stdout: "tag": "", 2026-02-17T21:22:20.641 INFO:tasks.workunit.client.0.trial031.stdout: "health": { 2026-02-17T21:22:20.641 INFO:tasks.workunit.client.0.trial031.stdout: "status": "HEALTH_WARN", 2026-02-17T21:22:20.641 INFO:tasks.workunit.client.0.trial031.stdout: "checks": { 2026-02-17T21:22:20.641 INFO:tasks.workunit.client.0.trial031.stdout: "OSDMAP_FLAGS": { 2026-02-17T21:22:20.641 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:22:20.641 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:22:20.641 INFO:tasks.workunit.client.0.trial031.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-17T21:22:20.641 INFO:tasks.workunit.client.0.trial031.stdout: "count": 17 2026-02-17T21:22:20.642 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.642 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [], 2026-02-17T21:22:20.642 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:22:20.642 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.642 INFO:tasks.workunit.client.0.trial031.stdout: "OSD_DOWN": { 2026-02-17T21:22:20.642 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:22:20.642 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:22:20.642 INFO:tasks.workunit.client.0.trial031.stdout: "message": "1 osds down", 2026-02-17T21:22:20.642 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:22:20.642 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.642 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:22:20.642 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.643 INFO:tasks.workunit.client.0.trial031.stdout: "message": "osd.1 (root=default,host=trial031) is down" 2026-02-17T21:22:20.643 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.643 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.643 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:22:20.643 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.643 INFO:tasks.workunit.client.0.trial031.stdout: "PG_DEGRADED": { 2026-02-17T21:22:20.643 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:22:20.643 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:22:20.643 INFO:tasks.workunit.client.0.trial031.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 32 pgs degraded", 2026-02-17T21:22:20.643 INFO:tasks.workunit.client.0.trial031.stdout: "count": 32 2026-02-17T21:22:20.643 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.643 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:22:20.643 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.644 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.644 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.644 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.644 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.1 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.644 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.644 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.644 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.2 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.644 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.644 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.644 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.3 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.644 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.644 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.645 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.4 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.645 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.645 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.645 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.5 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.645 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.645 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.645 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.6 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.645 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.645 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.645 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.7 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.645 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.645 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.645 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.8 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.646 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.646 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.646 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.9 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.646 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.646 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.646 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.a is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.646 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.646 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.646 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.b is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.646 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.646 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.646 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.c is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.646 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.647 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.647 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.d is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.647 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.647 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.647 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.e is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.647 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.647 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.647 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.f is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.647 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.647 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.647 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.10 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.647 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.647 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.648 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.11 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.648 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.648 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.648 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.12 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.648 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.648 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.648 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.13 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.648 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.648 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.648 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.14 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.648 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.648 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.648 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.15 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.649 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.649 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.649 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.16 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.649 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.649 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.649 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.17 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.649 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.649 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.649 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.18 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.649 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.649 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.649 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.19 is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.649 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.650 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.650 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.1a is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.650 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.650 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.650 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.1b is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.650 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.650 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.650 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.1c is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.650 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.650 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.650 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.1d is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.650 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.650 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.651 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.1e is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.651 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.651 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.651 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.1f is active+undersized+degraded, acting [0,2]" 2026-02-17T21:22:20.651 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.651 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.651 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:22:20.651 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.651 INFO:tasks.workunit.client.0.trial031.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-17T21:22:20.651 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:22:20.651 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:22:20.651 INFO:tasks.workunit.client.0.trial031.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-17T21:22:20.651 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:22:20.652 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.652 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:22:20.652 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.652 INFO:tasks.workunit.client.0.trial031.stdout: "message": "application not enabled on pool 'test'" 2026-02-17T21:22:20.652 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.652 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.652 INFO:tasks.workunit.client.0.trial031.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-17T21:22:20.652 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.652 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.652 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:22:20.652 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.652 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.652 INFO:tasks.workunit.client.0.trial031.stdout: "mutes": [] 2026-02-17T21:22:20.653 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.653 INFO:tasks.workunit.client.0.trial031.stdout: "monmap_first_committed": 1, 2026-02-17T21:22:20.653 INFO:tasks.workunit.client.0.trial031.stdout: "monmap_last_committed": 1, 2026-02-17T21:22:20.653 INFO:tasks.workunit.client.0.trial031.stdout: "monmap": { 2026-02-17T21:22:20.653 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 1, 2026-02-17T21:22:20.653 INFO:tasks.workunit.client.0.trial031.stdout: "fsid": "5ebac6e7-5321-4893-ad68-c06127ed455b", 2026-02-17T21:22:20.653 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:20:44.224453Z", 2026-02-17T21:22:20.653 INFO:tasks.workunit.client.0.trial031.stdout: "created": "2026-02-17T21:20:44.224453Z", 2026-02-17T21:22:20.653 INFO:tasks.workunit.client.0.trial031.stdout: "min_mon_release": 19, 2026-02-17T21:22:20.653 INFO:tasks.workunit.client.0.trial031.stdout: "min_mon_release_name": "squid", 2026-02-17T21:22:20.653 INFO:tasks.workunit.client.0.trial031.stdout: "election_strategy": 1, 2026-02-17T21:22:20.653 INFO:tasks.workunit.client.0.trial031.stdout: "disallowed_leaders": "", 2026-02-17T21:22:20.653 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode": false, 2026-02-17T21:22:20.654 INFO:tasks.workunit.client.0.trial031.stdout: "tiebreaker_mon": "", 2026-02-17T21:22:20.654 INFO:tasks.workunit.client.0.trial031.stdout: "removed_ranks": "", 2026-02-17T21:22:20.654 INFO:tasks.workunit.client.0.trial031.stdout: "features": { 2026-02-17T21:22:20.654 INFO:tasks.workunit.client.0.trial031.stdout: "persistent": [ 2026-02-17T21:22:20.654 INFO:tasks.workunit.client.0.trial031.stdout: "kraken", 2026-02-17T21:22:20.654 INFO:tasks.workunit.client.0.trial031.stdout: "luminous", 2026-02-17T21:22:20.654 INFO:tasks.workunit.client.0.trial031.stdout: "mimic", 2026-02-17T21:22:20.654 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap-prune", 2026-02-17T21:22:20.654 INFO:tasks.workunit.client.0.trial031.stdout: "nautilus", 2026-02-17T21:22:20.654 INFO:tasks.workunit.client.0.trial031.stdout: "octopus", 2026-02-17T21:22:20.654 INFO:tasks.workunit.client.0.trial031.stdout: "pacific", 2026-02-17T21:22:20.654 INFO:tasks.workunit.client.0.trial031.stdout: "elector-pinging", 2026-02-17T21:22:20.654 INFO:tasks.workunit.client.0.trial031.stdout: "quincy", 2026-02-17T21:22:20.654 INFO:tasks.workunit.client.0.trial031.stdout: "reef", 2026-02-17T21:22:20.655 INFO:tasks.workunit.client.0.trial031.stdout: "squid" 2026-02-17T21:22:20.655 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.655 INFO:tasks.workunit.client.0.trial031.stdout: "optional": [] 2026-02-17T21:22:20.655 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.655 INFO:tasks.workunit.client.0.trial031.stdout: "mons": [ 2026-02-17T21:22:20.655 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.655 INFO:tasks.workunit.client.0.trial031.stdout: "rank": 0, 2026-02-17T21:22:20.655 INFO:tasks.workunit.client.0.trial031.stdout: "name": "a", 2026-02-17T21:22:20.655 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:22:20.655 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:20.655 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.655 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:20.655 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:7115", 2026-02-17T21:22:20.656 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 0 2026-02-17T21:22:20.656 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.656 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.656 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.656 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:7115/0", 2026-02-17T21:22:20.656 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-17T21:22:20.656 INFO:tasks.workunit.client.0.trial031.stdout: "priority": 0, 2026-02-17T21:22:20.656 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 0, 2026-02-17T21:22:20.656 INFO:tasks.workunit.client.0.trial031.stdout: "crush_location": "{}" 2026-02-17T21:22:20.656 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.656 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.656 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.656 INFO:tasks.workunit.client.0.trial031.stdout: "quorum": [ 2026-02-17T21:22:20.657 INFO:tasks.workunit.client.0.trial031.stdout: 0 2026-02-17T21:22:20.657 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.657 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap": { 2026-02-17T21:22:20.657 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 45, 2026-02-17T21:22:20.657 INFO:tasks.workunit.client.0.trial031.stdout: "fsid": "5ebac6e7-5321-4893-ad68-c06127ed455b", 2026-02-17T21:22:20.657 INFO:tasks.workunit.client.0.trial031.stdout: "created": "2026-02-17T21:20:44.301417+0000", 2026-02-17T21:22:20.657 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:21:50.541606+0000", 2026-02-17T21:22:20.657 INFO:tasks.workunit.client.0.trial031.stdout: "last_up_change": "2026-02-17T21:21:49.388629+0000", 2026-02-17T21:22:20.657 INFO:tasks.workunit.client.0.trial031.stdout: "last_in_change": "2026-02-17T21:21:50.439368+0000", 2026-02-17T21:22:20.657 INFO:tasks.workunit.client.0.trial031.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-17T21:22:20.657 INFO:tasks.workunit.client.0.trial031.stdout: "flags_num": 5800384, 2026-02-17T21:22:20.657 INFO:tasks.workunit.client.0.trial031.stdout: "flags_set": [ 2026-02-17T21:22:20.657 INFO:tasks.workunit.client.0.trial031.stdout: "nodown", 2026-02-17T21:22:20.658 INFO:tasks.workunit.client.0.trial031.stdout: "noin", 2026-02-17T21:22:20.658 INFO:tasks.workunit.client.0.trial031.stdout: "noout", 2026-02-17T21:22:20.658 INFO:tasks.workunit.client.0.trial031.stdout: "pglog_hardlimit", 2026-02-17T21:22:20.658 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snapdirs", 2026-02-17T21:22:20.658 INFO:tasks.workunit.client.0.trial031.stdout: "recovery_deletes", 2026-02-17T21:22:20.658 INFO:tasks.workunit.client.0.trial031.stdout: "sortbitwise" 2026-02-17T21:22:20.658 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.658 INFO:tasks.workunit.client.0.trial031.stdout: "crush_version": 9, 2026-02-17T21:22:20.658 INFO:tasks.workunit.client.0.trial031.stdout: "full_ratio": 0.99000000953674316, 2026-02-17T21:22:20.658 INFO:tasks.workunit.client.0.trial031.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-17T21:22:20.658 INFO:tasks.workunit.client.0.trial031.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-17T21:22:20.658 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_snapshot": "", 2026-02-17T21:22:20.658 INFO:tasks.workunit.client.0.trial031.stdout: "pool_max": 1, 2026-02-17T21:22:20.659 INFO:tasks.workunit.client.0.trial031.stdout: "max_osd": 3, 2026-02-17T21:22:20.659 INFO:tasks.workunit.client.0.trial031.stdout: "require_min_compat_client": "luminous", 2026-02-17T21:22:20.659 INFO:tasks.workunit.client.0.trial031.stdout: "min_compat_client": "jewel", 2026-02-17T21:22:20.659 INFO:tasks.workunit.client.0.trial031.stdout: "require_osd_release": "squid", 2026-02-17T21:22:20.659 INFO:tasks.workunit.client.0.trial031.stdout: "allow_crimson": false, 2026-02-17T21:22:20.659 INFO:tasks.workunit.client.0.trial031.stdout: "pools": [ 2026-02-17T21:22:20.659 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.659 INFO:tasks.workunit.client.0.trial031.stdout: "pool": 1, 2026-02-17T21:22:20.659 INFO:tasks.workunit.client.0.trial031.stdout: "pool_name": "test", 2026-02-17T21:22:20.659 INFO:tasks.workunit.client.0.trial031.stdout: "create_time": "2026-02-17T21:21:13.412557+0000", 2026-02-17T21:22:20.659 INFO:tasks.workunit.client.0.trial031.stdout: "flags": 1, 2026-02-17T21:22:20.659 INFO:tasks.workunit.client.0.trial031.stdout: "flags_names": "hashpspool", 2026-02-17T21:22:20.659 INFO:tasks.workunit.client.0.trial031.stdout: "type": 1, 2026-02-17T21:22:20.660 INFO:tasks.workunit.client.0.trial031.stdout: "size": 3, 2026-02-17T21:22:20.660 INFO:tasks.workunit.client.0.trial031.stdout: "min_size": 2, 2026-02-17T21:22:20.660 INFO:tasks.workunit.client.0.trial031.stdout: "crush_rule": 0, 2026-02-17T21:22:20.660 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_count": 0, 2026-02-17T21:22:20.660 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_target": 0, 2026-02-17T21:22:20.660 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_barrier": 0, 2026-02-17T21:22:20.660 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-17T21:22:20.660 INFO:tasks.workunit.client.0.trial031.stdout: "is_stretch_pool": false, 2026-02-17T21:22:20.660 INFO:tasks.workunit.client.0.trial031.stdout: "object_hash": 2, 2026-02-17T21:22:20.660 INFO:tasks.workunit.client.0.trial031.stdout: "pg_autoscale_mode": "on", 2026-02-17T21:22:20.660 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num": 32, 2026-02-17T21:22:20.660 INFO:tasks.workunit.client.0.trial031.stdout: "pg_placement_num": 1, 2026-02-17T21:22:20.660 INFO:tasks.workunit.client.0.trial031.stdout: "pg_placement_num_target": 32, 2026-02-17T21:22:20.661 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num_target": 32, 2026-02-17T21:22:20.661 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num_pending": 32, 2026-02-17T21:22:20.661 INFO:tasks.workunit.client.0.trial031.stdout: "last_pg_merge_meta": { 2026-02-17T21:22:20.661 INFO:tasks.workunit.client.0.trial031.stdout: "source_pgid": "0.0", 2026-02-17T21:22:20.661 INFO:tasks.workunit.client.0.trial031.stdout: "ready_epoch": 0, 2026-02-17T21:22:20.661 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_started": 0, 2026-02-17T21:22:20.661 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_clean": 0, 2026-02-17T21:22:20.661 INFO:tasks.workunit.client.0.trial031.stdout: "source_version": "0'0", 2026-02-17T21:22:20.661 INFO:tasks.workunit.client.0.trial031.stdout: "target_version": "0'0" 2026-02-17T21:22:20.661 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.661 INFO:tasks.workunit.client.0.trial031.stdout: "last_change": "38", 2026-02-17T21:22:20.661 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend": "0", 2026-02-17T21:22:20.661 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-17T21:22:20.662 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend_preluminous": "38", 2026-02-17T21:22:20.662 INFO:tasks.workunit.client.0.trial031.stdout: "auid": 0, 2026-02-17T21:22:20.662 INFO:tasks.workunit.client.0.trial031.stdout: "snap_mode": "selfmanaged", 2026-02-17T21:22:20.662 INFO:tasks.workunit.client.0.trial031.stdout: "snap_seq": 0, 2026-02-17T21:22:20.662 INFO:tasks.workunit.client.0.trial031.stdout: "snap_epoch": 0, 2026-02-17T21:22:20.662 INFO:tasks.workunit.client.0.trial031.stdout: "pool_snaps": [], 2026-02-17T21:22:20.662 INFO:tasks.workunit.client.0.trial031.stdout: "removed_snaps": "[]", 2026-02-17T21:22:20.662 INFO:tasks.workunit.client.0.trial031.stdout: "quota_max_bytes": 0, 2026-02-17T21:22:20.662 INFO:tasks.workunit.client.0.trial031.stdout: "quota_max_objects": 0, 2026-02-17T21:22:20.662 INFO:tasks.workunit.client.0.trial031.stdout: "tiers": [], 2026-02-17T21:22:20.662 INFO:tasks.workunit.client.0.trial031.stdout: "tier_of": -1, 2026-02-17T21:22:20.662 INFO:tasks.workunit.client.0.trial031.stdout: "read_tier": -1, 2026-02-17T21:22:20.662 INFO:tasks.workunit.client.0.trial031.stdout: "write_tier": -1, 2026-02-17T21:22:20.663 INFO:tasks.workunit.client.0.trial031.stdout: "cache_mode": "none", 2026-02-17T21:22:20.663 INFO:tasks.workunit.client.0.trial031.stdout: "target_max_bytes": 0, 2026-02-17T21:22:20.663 INFO:tasks.workunit.client.0.trial031.stdout: "target_max_objects": 0, 2026-02-17T21:22:20.663 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-17T21:22:20.663 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-17T21:22:20.663 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-17T21:22:20.663 INFO:tasks.workunit.client.0.trial031.stdout: "cache_min_flush_age": 0, 2026-02-17T21:22:20.663 INFO:tasks.workunit.client.0.trial031.stdout: "cache_min_evict_age": 0, 2026-02-17T21:22:20.663 INFO:tasks.workunit.client.0.trial031.stdout: "erasure_code_profile": "", 2026-02-17T21:22:20.663 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_params": { 2026-02-17T21:22:20.663 INFO:tasks.workunit.client.0.trial031.stdout: "type": "none" 2026-02-17T21:22:20.663 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.663 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_period": 0, 2026-02-17T21:22:20.663 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_count": 0, 2026-02-17T21:22:20.664 INFO:tasks.workunit.client.0.trial031.stdout: "use_gmt_hitset": true, 2026-02-17T21:22:20.664 INFO:tasks.workunit.client.0.trial031.stdout: "min_read_recency_for_promote": 0, 2026-02-17T21:22:20.664 INFO:tasks.workunit.client.0.trial031.stdout: "min_write_recency_for_promote": 0, 2026-02-17T21:22:20.664 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_grade_decay_rate": 0, 2026-02-17T21:22:20.664 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_search_last_n": 0, 2026-02-17T21:22:20.664 INFO:tasks.workunit.client.0.trial031.stdout: "grade_table": [], 2026-02-17T21:22:20.664 INFO:tasks.workunit.client.0.trial031.stdout: "stripe_width": 0, 2026-02-17T21:22:20.664 INFO:tasks.workunit.client.0.trial031.stdout: "expected_num_objects": 0, 2026-02-17T21:22:20.664 INFO:tasks.workunit.client.0.trial031.stdout: "fast_read": false, 2026-02-17T21:22:20.664 INFO:tasks.workunit.client.0.trial031.stdout: "options": {}, 2026-02-17T21:22:20.664 INFO:tasks.workunit.client.0.trial031.stdout: "application_metadata": {}, 2026-02-17T21:22:20.664 INFO:tasks.workunit.client.0.trial031.stdout: "read_balance": { 2026-02-17T21:22:20.664 INFO:tasks.workunit.client.0.trial031.stdout: "score_type": "Fair distribution", 2026-02-17T21:22:20.665 INFO:tasks.workunit.client.0.trial031.stdout: "score_acting": 2.9900000095367432, 2026-02-17T21:22:20.665 INFO:tasks.workunit.client.0.trial031.stdout: "score_stable": 2.9900000095367432, 2026-02-17T21:22:20.665 INFO:tasks.workunit.client.0.trial031.stdout: "optimal_score": 0.67000001668930054, 2026-02-17T21:22:20.665 INFO:tasks.workunit.client.0.trial031.stdout: "raw_score_acting": 2, 2026-02-17T21:22:20.665 INFO:tasks.workunit.client.0.trial031.stdout: "raw_score_stable": 2, 2026-02-17T21:22:20.665 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity_weighted": 1, 2026-02-17T21:22:20.665 INFO:tasks.workunit.client.0.trial031.stdout: "average_primary_affinity": 1, 2026-02-17T21:22:20.665 INFO:tasks.workunit.client.0.trial031.stdout: "average_primary_affinity_weighted": 1 2026-02-17T21:22:20.665 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.665 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.665 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.665 INFO:tasks.workunit.client.0.trial031.stdout: "osds": [ 2026-02-17T21:22:20.666 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.666 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:22:20.666 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "c804922e-7eb2-4582-9dc2-d361d3e2c25a", 2026-02-17T21:22:20.666 INFO:tasks.workunit.client.0.trial031.stdout: "up": 1, 2026-02-17T21:22:20.666 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:22:20.666 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:22:20.666 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:22:20.666 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 5, 2026-02-17T21:22:20.666 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 28, 2026-02-17T21:22:20.666 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 35, 2026-02-17T21:22:20.666 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 44, 2026-02-17T21:22:20.666 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 29, 2026-02-17T21:22:20.666 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:22:20.666 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:22:20.667 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:20.667 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.667 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:20.667 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6802", 2026-02-17T21:22:20.667 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:20.667 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.667 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.667 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:20.667 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6803", 2026-02-17T21:22:20.667 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:20.667 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.667 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.667 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.668 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:22:20.668 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:20.668 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.668 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:20.668 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6804", 2026-02-17T21:22:20.668 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:20.668 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.668 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.668 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:20.668 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6805", 2026-02-17T21:22:20.668 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:20.668 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.668 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.669 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.669 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:22:20.669 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:20.669 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.669 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:20.669 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6808", 2026-02-17T21:22:20.669 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:20.669 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.669 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.669 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:20.669 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6809", 2026-02-17T21:22:20.669 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:20.669 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.670 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.670 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.670 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:22:20.670 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:20.670 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.670 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:20.670 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6806", 2026-02-17T21:22:20.670 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:20.670 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.670 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.670 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:20.670 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6807", 2026-02-17T21:22:20.671 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:20.671 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.671 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.671 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.671 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6803/3194159082", 2026-02-17T21:22:20.671 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6805/3194159082", 2026-02-17T21:22:20.671 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6809/3194159082", 2026-02-17T21:22:20.671 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6807/3194159082", 2026-02-17T21:22:20.671 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:22:20.671 INFO:tasks.workunit.client.0.trial031.stdout: "exists", 2026-02-17T21:22:20.671 INFO:tasks.workunit.client.0.trial031.stdout: "up" 2026-02-17T21:22:20.671 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.672 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.672 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.672 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:22:20.672 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "2bdda522-e523-463c-b285-abb08be7107e", 2026-02-17T21:22:20.672 INFO:tasks.workunit.client.0.trial031.stdout: "up": 0, 2026-02-17T21:22:20.672 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:22:20.672 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:22:20.672 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:22:20.672 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 0, 2026-02-17T21:22:20.672 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 0, 2026-02-17T21:22:20.672 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 9, 2026-02-17T21:22:20.672 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 21, 2026-02-17T21:22:20.672 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 30, 2026-02-17T21:22:20.672 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:22:20.673 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:22:20.673 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:20.673 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.673 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:20.673 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6810", 2026-02-17T21:22:20.673 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:20.673 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.673 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.673 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:20.673 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6811", 2026-02-17T21:22:20.673 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:20.673 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.673 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.674 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.674 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:22:20.674 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:20.674 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.674 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:20.674 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6812", 2026-02-17T21:22:20.674 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:20.674 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.674 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.674 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:20.674 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6813", 2026-02-17T21:22:20.674 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:20.674 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.674 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.675 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.675 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:22:20.675 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:20.675 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.675 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:20.675 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6816", 2026-02-17T21:22:20.675 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:20.675 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.675 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.675 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:20.675 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6817", 2026-02-17T21:22:20.675 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:20.675 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.675 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.675 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.676 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:22:20.676 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:20.676 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.676 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:20.676 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6814", 2026-02-17T21:22:20.676 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:20.676 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.676 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.676 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:20.676 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6815", 2026-02-17T21:22:20.676 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:20.676 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.676 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.676 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.677 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6811/376030295", 2026-02-17T21:22:20.677 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6813/376030295", 2026-02-17T21:22:20.677 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6817/376030295", 2026-02-17T21:22:20.677 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6815/376030295", 2026-02-17T21:22:20.677 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:22:20.677 INFO:tasks.workunit.client.0.trial031.stdout: "exists" 2026-02-17T21:22:20.677 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.677 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.677 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.677 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:22:20.677 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "258ca9b7-06c7-4314-ac32-bb7e4d29e018", 2026-02-17T21:22:20.677 INFO:tasks.workunit.client.0.trial031.stdout: "up": 1, 2026-02-17T21:22:20.677 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:22:20.677 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 13, 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 28, 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 40, 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 0, 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 31, 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6810", 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6811", 2026-02-17T21:22:20.678 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6812", 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6813", 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.679 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6816", 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6817", 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:20.680 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6814", 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6815", 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6811/1416133595", 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6813/1416133595", 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6817/1416133595", 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6815/1416133595", 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: "exists", 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: "up" 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.681 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: "osd_xinfo": [ 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:21:38.109278+0000", 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "2026-02-17T21:20:53.042100+0000", 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:21:38.381347+0000", 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:22:20.682 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:21:38.685698+0000", 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "2026-02-17T21:21:11.695133+0000", 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap": [], 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap_items": [], 2026-02-17T21:22:20.683 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap_primaries": [], 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "pg_temp": [], 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "primary_temp": [], 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "blocklist": {}, 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "range_blocklist": {}, 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "erasure_code_profiles": { 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "default": { 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "k": "2", 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "m": "2", 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "plugin": "jerasure", 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "technique": "reed_sol_van" 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "removed_snaps_queue": [], 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "new_removed_snaps": [], 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "new_purged_snaps": [], 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "crush_node_flags": {}, 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "device_class_flags": {}, 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode": { 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode_enabled": false, 2026-02-17T21:22:20.684 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_bucket_count": 0, 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "degraded_stretch_mode": 0, 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "recovering_stretch_mode": 0, 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode_bucket": 0 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "osd_metadata": [ 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6804/3194159082,v1:127.0.0.1:6805/3194159082]", 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:22:20.685 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:20:46.458290Z", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:22:20.686 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6802/3194159082,v1:127.0.0.1:6803/3194159082]", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/3194159082,v1:127.0.0.1:6809/3194159082]", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/3194159082,v1:127.0.0.1:6807/3194159082]", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/0", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.687 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6812/376030295,v1:127.0.0.1:6813/376030295]", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:22:20.688 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:20:55.680906Z", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6810/376030295,v1:127.0.0.1:6811/376030295]", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/376030295,v1:127.0.0.1:6817/376030295]", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/376030295,v1:127.0.0.1:6815/376030295]", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:22:20.689 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/1", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6812/1416133595,v1:127.0.0.1:6813/1416133595]", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:22:20.690 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:21:05.125195Z", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6810/1416133595,v1:127.0.0.1:6811/1416133595]", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/1416133595,v1:127.0.0.1:6817/1416133595]", 2026-02-17T21:22:20.691 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/1416133595,v1:127.0.0.1:6815/1416133595]", 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/2", 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_clean_epochs": { 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "min_last_epoch_clean": 0, 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_clean": { 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "per_pool": [] 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: "osd_epochs": [ 2026-02-17T21:22:20.692 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 35 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 42 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_first_committed": 1, 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_last_committed": 45, 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: "crushmap": { 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: "devices": [ 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.0", 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.1", 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:22:20.693 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.2", 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: "types": [ 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 0, 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd" 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: "name": "host" 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 2, 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: "name": "chassis" 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 3, 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: "name": "rack" 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.694 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 4, 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: "name": "row" 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 5, 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: "name": "pdu" 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 6, 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: "name": "pod" 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 7, 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: "name": "room" 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 8, 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: "name": "datacenter" 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.695 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stderr:report 1203606100 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 9, 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: "name": "zone" 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 10, 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: "name": "region" 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: "name": "root" 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: "buckets": [ 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: "id": -1, 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: "name": "default", 2026-02-17T21:22:20.696 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "root", 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "id": -3, 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "id": -2, 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "name": "default~ssd", 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "root", 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "id": -4, 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:22:20.697 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "id": -3, 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "name": "trial031", 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "host", 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 1 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 2 2026-02-17T21:22:20.698 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "id": -4, 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "name": "trial031~ssd", 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "host", 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 1 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 2 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.699 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: "rules": [ 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: "rule_id": 0, 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: "rule_name": "replicated_rule", 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: "type": 1, 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: "steps": [ 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: "op": "take", 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: "item": -1, 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: "item_name": "default" 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: "op": "choose_firstn", 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: "num": 0, 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: "type": "osd" 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: "op": "emit" 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: "tunables": { 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: "choose_local_tries": 0, 2026-02-17T21:22:20.700 INFO:tasks.workunit.client.0.trial031.stdout: "choose_local_fallback_tries": 0, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "choose_total_tries": 50, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_descend_once": 1, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_vary_r": 1, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_stable": 1, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "msr_descents": 100, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "msr_collision_tries": 100, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "straw_calc_version": 1, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "allowed_bucket_algs": 54, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "profile": "jewel", 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "optimal_tunables": 1, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "legacy_tunables": 0, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "minimum_required_version": "jewel", 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables": 1, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables2": 1, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "has_v2_rules": 0, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables3": 1, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "has_v3_rules": 0, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "has_v4_buckets": 1, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables5": 1, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "has_v5_rules": 0, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "has_msr_rules": 0 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "choose_args": {} 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "fsmap": { 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 1, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "btime": "2026-02-17T21:20:44:301188+0000", 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "default_fscid": -1, 2026-02-17T21:22:20.701 INFO:tasks.workunit.client.0.trial031.stdout: "compat": { 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "compat": {}, 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "ro_compat": {}, 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "incompat": { 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "feature_1": "base v0.20", 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "feature_2": "client writeable ranges", 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "feature_3": "default file layouts on dirs", 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "feature_4": "dir inode in separate object", 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "feature_5": "mds uses versioned encoding", 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "feature_8": "no anchor table", 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "feature_9": "file layout v2", 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "feature_10": "snaprealm v2", 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "feature_11": "minor log segments", 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "feature_12": "quiesce subvolumes" 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "feature_flags": { 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "enable_multiple": true, 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "ever_enabled_multiple": true 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "standbys": [], 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "filesystems": [] 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "mdsmap_first_committed": 1, 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "mdsmap_last_committed": 1, 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "auth": { 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "first_committed": 1, 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "last_committed": 5, 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: "num_secrets": 3 2026-02-17T21:22:20.702 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg": 32, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_active": 32, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_unknown": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_osd": 3, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "pool_sum": { 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "stat_sum": { 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes": 70600, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects": 100, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_clones": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_copies": 300, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_degraded": 100, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_misplaced": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_unfound": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_dirty": 100, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_whiteouts": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_read": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_read_kb": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_write": 100, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_write_kb": 100, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_scrub_errors": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_deep_scrub_errors": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_recovered": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_recovered": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_keys_recovered": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_omap": 0, 2026-02-17T21:22:20.703 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_kb": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_kb": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_promote": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_high": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_low": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_some": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_full": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_pinned": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_legacy_snapsets": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_large_omap_objects": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_manifest": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_bytes": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_keys": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_repaired": 0 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "store_stats": { 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "total": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "available": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 0, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 0 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "log_size": 512, 2026-02-17T21:22:20.704 INFO:tasks.workunit.client.0.trial031.stdout: "ondisk_log_size": 512, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "up": 64, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "acting": 64, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "num_store_stats": 0 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "osd_sum": { 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 0, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 0, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "num_pgs": 65, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "num_osds": 3, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_osds": 3, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "kb": 314572800, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used": 83992, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_data": 2160, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_omap": 10, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_meta": 81717, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "kb_avail": 314488808, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "statfs": { 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "total": 322122547200, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "available": 322036539392, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 2211840, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 519656, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 10389, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 83679083 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "hb_peers": [], 2026-02-17T21:22:20.705 INFO:tasks.workunit.client.0.trial031.stdout: "snap_trim_queue_len": 0, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "num_snap_trimming": 0, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "num_shards_repaired": 0, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "op_queue_age_hist": { 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "histogram": [], 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "upper_bound": 1 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "perf_stat": { 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ms": 0, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ms": 0, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ns": 0, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ns": 0 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "alerts": [], 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "network_ping_times": [] 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "osd_sum_by_class": { 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "ssd": { 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 0, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 0, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "num_pgs": 65, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "num_osds": 3, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_osds": 3, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "kb": 314572800, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used": 83992, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_data": 2160, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_omap": 10, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_meta": 81717, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "kb_avail": 314488808, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "statfs": { 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "total": 322122547200, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "available": 322036539392, 2026-02-17T21:22:20.706 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 2211840, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 519656, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 10389, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 83679083 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "hb_peers": [], 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "snap_trim_queue_len": 0, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "num_snap_trimming": 0, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "num_shards_repaired": 0, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "op_queue_age_hist": { 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "histogram": [], 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "upper_bound": 1 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "perf_stat": { 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ms": 0, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ms": 0, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ns": 0, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ns": 0 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "alerts": [], 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "network_ping_times": [] 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "pool_stats": [ 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "poolid": 1, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg": 32, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "stat_sum": { 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes": 70600, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects": 100, 2026-02-17T21:22:20.707 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_clones": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_copies": 300, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_degraded": 100, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_misplaced": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_unfound": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_dirty": 100, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_whiteouts": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_read": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_read_kb": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_write": 100, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_write_kb": 100, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_scrub_errors": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_deep_scrub_errors": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_recovered": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_recovered": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_keys_recovered": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_omap": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_kb": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_kb": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_promote": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_high": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_low": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_some": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_full": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_pinned": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_legacy_snapsets": 0, 2026-02-17T21:22:20.708 INFO:tasks.workunit.client.0.trial031.stdout: "num_large_omap_objects": 0, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_manifest": 0, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_bytes": 0, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_keys": 0, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_repaired": 0 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "store_stats": { 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "total": 0, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "available": 0, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 1220608, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 210910, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 0, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 0 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "log_size": 512, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "ondisk_log_size": 512, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "up": 64, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "acting": 64, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "num_store_stats": 3 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "osd_stats": [ 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 150323855369 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 38654705678 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.709 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 171798691848 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_by_state": [ 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "state": "active+undersized+degraded", 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "num": 32 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_by_osd": [ 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 32, 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 32, 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 32, 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 0, 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snaps": [ 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "pool": 1, 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snaps": {} 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.710 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "servicemap": { 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 4, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:22:02.761619+0000", 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "services": { 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "osd": { 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "daemons": { 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "summary": "", 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "0": { 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "start_epoch": 0, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "start_stamp": "0.000000", 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "gid": 0, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "metadata": {}, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "task_status": {} 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "2": { 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "start_epoch": 0, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "start_stamp": "0.000000", 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "gid": 0, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "metadata": {}, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "task_status": {} 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "mgrstat_first_committed": 1, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "mgrstat_last_committed": 45, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "logm_first_committed": 1, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "logm_last_committed": 249, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "paxos": { 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "first_committed": 1, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "last_committed": 348, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "last_pn": 0, 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: "accepted_pn": 0 2026-02-17T21:22:20.711 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:20.712 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:22:20.712 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-17T21:22:20.712 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:753: TEST_divergent_3: expr 2 + 2 2026-02-17T21:22:20.712 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:753: TEST_divergent_3: objname=existing_4 2026-02-17T21:22:20.712 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:754: TEST_divergent_3: echo 'writing non-divergent object existing_4' 2026-02-17T21:22:20.712 INFO:tasks.workunit.client.0.trial031.stdout:writing non-divergent object existing_4 2026-02-17T21:22:20.712 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:755: TEST_divergent_3: ceph pg dump pgs 2026-02-17T21:22:20.841 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:22:20.841 INFO:tasks.workunit.client.0.trial031.stdout:1.1f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343954+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796405+0000 0 0 2026-02-17T21:22:20.841 INFO:tasks.workunit.client.0.trial031.stdout:1.1e 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341401+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796444+0000 0 0 2026-02-17T21:22:20.841 INFO:tasks.workunit.client.0.trial031.stdout:1.1d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341205+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796363+0000 0 0 2026-02-17T21:22:20.841 INFO:tasks.workunit.client.0.trial031.stdout:1.b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341470+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796285+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341090+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796174+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.9 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341294+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796222+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.8 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.340967+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796205+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.7 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344971+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796303+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.6 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343790+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796248+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.5 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343964+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796257+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.4 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343986+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796276+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.3 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343765+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796266+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.2 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343766+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796231+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343947+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796293+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.1 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341729+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796196+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.1c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341106+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796421+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.0 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341066+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796213+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341537+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796240+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.e 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343911+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796396+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344345+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796355+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.10 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341474+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796388+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.11 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342237+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796312+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.12 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341306+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796337+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.13 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341120+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796429+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.14 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341229+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796436+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.15 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341182+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796346+0000 0 0 2026-02-17T21:22:20.842 INFO:tasks.workunit.client.0.trial031.stdout:1.16 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341319+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796453+0000 0 0 2026-02-17T21:22:20.843 INFO:tasks.workunit.client.0.trial031.stdout:1.17 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343856+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796413+0000 0 0 2026-02-17T21:22:20.843 INFO:tasks.workunit.client.0.trial031.stdout:1.18 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342096+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796328+0000 0 0 2026-02-17T21:22:20.843 INFO:tasks.workunit.client.0.trial031.stdout:1.19 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343794+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796380+0000 0 0 2026-02-17T21:22:20.843 INFO:tasks.workunit.client.0.trial031.stdout:1.1a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342036+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796321+0000 0 0 2026-02-17T21:22:20.843 INFO:tasks.workunit.client.0.trial031.stdout:1.1b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341167+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796371+0000 0 0 2026-02-17T21:22:20.843 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:22:20.843 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:22:20.843 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:22:20.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:756: TEST_divergent_3: rados -p test put existing_4 /etc/resolv.conf 2026-02-17T21:22:20.879 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:758: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-17T21:22:20.879 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:758: TEST_divergent_3: wait_for_clean 2026-02-17T21:22:20.880 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:22:20.880 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:22:20.880 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:22:20.880 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-17T21:22:20.880 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:22:20.880 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:22:20.881 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:22:20.881 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:22:20.881 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:22:20.953 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-17T21:22:20.953 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:22:20.953 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:22:20.953 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:22:20.953 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:22:20.953 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:22:21.159 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:22:21.159 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:22:21.159 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:22:21.159 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:22:21.159 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:22:21.159 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:22:21.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855372 2026-02-17T21:22:21.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855372 2026-02-17T21:22:21.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855372' 2026-02-17T21:22:21.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:22:21.239 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:22:21.299 INFO:tasks.workunit.client.0.trial031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-17T21:22:21.303 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-17T21:22:21.303 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-17T21:22:21.303 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-17T21:22:21.303 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:22:21.303 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:22:21.381 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691851 2026-02-17T21:22:21.381 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691851 2026-02-17T21:22:21.381 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855372 2-171798691851' 2026-02-17T21:22:21.381 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:22:21.381 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855372 2026-02-17T21:22:21.381 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:22:21.382 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:22:21.383 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855372 2026-02-17T21:22:21.383 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:22:21.384 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855372 2026-02-17T21:22:21.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855372' 2026-02-17T21:22:21.385 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 150323855372 2026-02-17T21:22:21.385 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:22:21.588 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855370 -lt 150323855372 2026-02-17T21:22:21.588 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:22:22.589 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:22:22.589 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:22:22.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855370 -lt 150323855372 2026-02-17T21:22:22.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:22:23.796 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T21:22:23.797 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:22:24.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855372 -lt 150323855372 2026-02-17T21:22:24.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:22:24.001 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-171798691851 2026-02-17T21:22:24.001 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:22:24.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:22:24.003 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-171798691851 2026-02-17T21:22:24.003 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:22:24.004 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691851 2026-02-17T21:22:24.005 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 171798691851' 2026-02-17T21:22:24.005 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 171798691851 2026-02-17T21:22:24.005 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:22:24.214 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691851 -lt 171798691851 2026-02-17T21:22:24.214 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:22:24.215 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:22:24.215 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:22:24.488 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 == 0 2026-02-17T21:22:24.488 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:22:24.488 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:22:24.488 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:22:24.488 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:22:24.488 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:22:24.488 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:22:24.488 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:22:24.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:22:24.697 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:22:24.697 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:22:24.697 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:22:24.958 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:22:24.958 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T21:22:24.958 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T21:22:24.958 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T21:22:24.958 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:22:24.958 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T21:22:25.059 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:22:25.059 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:22:25.060 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:22:25.060 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:22:25.060 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:22:25.060 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:22:25.060 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:22:25.060 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:22:25.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:22:25.258 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:22:25.259 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:22:25.259 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:22:25.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:22:25.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:22:25.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:22:25.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:22:25.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:22:25.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:22:25.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:22:25.516 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:22:25.516 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:22:25.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:22:25.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:22:25.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-17T21:22:25.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:22:25.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T21:22:25.975 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:22:25.975 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:22:25.976 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:22:25.976 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:22:25.976 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:22:25.976 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:22:25.977 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:22:25.977 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:22:26.172 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:22:26.173 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:22:26.173 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:22:26.173 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:22:26.431 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:22:26.431 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:22:26.431 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:22:26.432 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:22:26.432 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:22:26.432 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:22:26.432 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:22:26.432 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:22:26.432 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:22:26.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:22:26.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:22:26.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-17T21:22:26.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:22:26.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T21:22:27.096 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:22:27.096 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:22:27.097 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:22:27.097 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:22:27.098 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:22:27.098 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:22:27.098 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:22:27.098 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:22:27.295 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:22:27.296 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:22:27.296 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:22:27.296 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:22:27.558 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:22:27.558 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:22:27.558 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:22:27.558 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:22:27.558 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:22:27.559 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:22:27.559 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:22:27.559 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:22:27.559 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:22:27.825 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:22:27.825 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:22:27.825 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-17T21:22:27.825 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:22:27.825 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T21:22:28.626 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:22:28.626 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:22:28.627 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:22:28.627 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:22:28.627 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:22:28.628 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:22:28.628 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:22:28.628 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:22:28.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:22:28.832 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:22:28.833 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:22:28.833 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:22:29.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:22:29.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:22:29.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:22:29.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:22:29.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:22:29.100 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:22:29.100 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:22:29.100 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:22:29.100 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:22:29.370 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:22:29.370 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:22:29.370 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-17T21:22:29.370 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:22:29.370 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-17T21:22:30.971 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:22:30.972 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:22:30.973 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:22:30.973 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:22:30.973 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:22:30.973 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:22:30.973 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:22:30.973 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:22:31.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:22:31.174 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:22:31.174 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:22:31.175 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:22:31.447 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:22:31.447 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:22:31.447 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:22:31.447 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:22:31.447 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:22:31.447 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:22:31.447 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:22:31.447 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:22:31.447 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:22:31.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:22:31.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:22:31.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-17T21:22:31.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:22:31.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-17T21:22:34.923 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:22:34.923 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:22:34.924 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:22:34.924 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:22:34.924 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:22:34.924 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:22:34.924 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:22:34.924 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:22:35.125 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:22:35.126 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:22:35.126 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:22:35.127 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:22:35.393 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:22:35.393 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:22:35.393 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:22:35.393 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:22:35.393 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:22:35.393 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:22:35.394 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:22:35.394 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:22:35.394 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:22:35.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:22:35.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:22:35.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-17T21:22:35.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:22:35.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-17T21:22:42.069 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:22:42.069 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:22:42.070 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:22:42.070 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:22:42.070 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:22:42.071 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:22:42.071 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:22:42.071 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:22:42.279 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:22:42.280 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:22:42.280 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:22:42.280 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:22:42.557 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:22:42.557 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:22:42.557 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:22:42.557 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:22:42.557 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:22:42.557 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:22:42.557 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:22:42.557 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:22:42.558 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:22:42.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:22:42.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:22:42.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-17T21:22:42.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:22:42.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-17T21:22:50.124 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:22:50.125 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:22:50.126 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:22:50.126 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:22:50.126 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:22:50.126 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:22:50.126 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:22:50.126 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:22:50.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:22:50.338 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:22:50.338 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:22:50.339 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:22:50.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-17T21:22:50.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:22:50.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:22:50.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:22:50.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:22:50.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:22:50.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:22:50.609 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:22:50.609 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:22:50.879 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:22:50.879 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:22:50.879 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-17T21:22:50.879 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-17T21:22:51.135 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:22:51.135 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_fingerprint": "f47ed877-51eb-480a-9d76-c2fb240ab7b0", 2026-02-17T21:22:51.135 INFO:tasks.workunit.client.0.trial031.stdout: "version": "19.2.3-679-gd855f53b", 2026-02-17T21:22:51.135 INFO:tasks.workunit.client.0.trial031.stdout: "commit": "d855f53b89fdcec760fd9232a5fb55ed4fb111a1", 2026-02-17T21:22:51.136 INFO:tasks.workunit.client.0.trial031.stdout: "timestamp": "2026-02-17T21:22:51.134275+0000", 2026-02-17T21:22:51.136 INFO:tasks.workunit.client.0.trial031.stdout: "tag": "", 2026-02-17T21:22:51.136 INFO:tasks.workunit.client.0.trial031.stdout: "health": { 2026-02-17T21:22:51.136 INFO:tasks.workunit.client.0.trial031.stdout: "status": "HEALTH_WARN", 2026-02-17T21:22:51.136 INFO:tasks.workunit.client.0.trial031.stdout: "checks": { 2026-02-17T21:22:51.136 INFO:tasks.workunit.client.0.trial031.stdout: "OSDMAP_FLAGS": { 2026-02-17T21:22:51.136 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:22:51.136 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:22:51.136 INFO:tasks.workunit.client.0.trial031.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-17T21:22:51.136 INFO:tasks.workunit.client.0.trial031.stdout: "count": 17 2026-02-17T21:22:51.136 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.136 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [], 2026-02-17T21:22:51.137 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:22:51.137 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.137 INFO:tasks.workunit.client.0.trial031.stdout: "OSD_DOWN": { 2026-02-17T21:22:51.137 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:22:51.137 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:22:51.137 INFO:tasks.workunit.client.0.trial031.stdout: "message": "1 osds down", 2026-02-17T21:22:51.137 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:22:51.137 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.137 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:22:51.137 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.137 INFO:tasks.workunit.client.0.trial031.stdout: "message": "osd.1 (root=default,host=trial031) is down" 2026-02-17T21:22:51.137 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.138 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.138 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:22:51.138 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.138 INFO:tasks.workunit.client.0.trial031.stdout: "PG_DEGRADED": { 2026-02-17T21:22:51.138 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:22:51.138 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:22:51.138 INFO:tasks.workunit.client.0.trial031.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 32 pgs degraded, 32 pgs undersized", 2026-02-17T21:22:51.138 INFO:tasks.workunit.client.0.trial031.stdout: "count": 64 2026-02-17T21:22:51.138 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.138 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:22:51.138 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.138 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.0 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.139 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.139 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.139 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.1 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.139 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.139 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.139 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.2 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.139 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.139 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.139 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.3 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.139 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.139 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.139 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.4 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.139 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.140 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.140 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.5 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.140 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.140 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.140 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.6 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.140 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.140 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.140 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.7 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.140 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.140 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.140 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.8 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.140 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.141 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.141 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.9 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.141 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.141 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.141 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.a is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.141 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.141 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.141 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.b is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.141 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.141 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.141 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.c is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.141 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.141 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.142 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.d is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.142 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.142 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.142 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.e is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.142 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.142 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.142 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.f is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.142 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.142 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.142 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.10 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.142 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.142 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.142 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.11 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.143 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.143 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.143 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.12 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.143 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.143 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.143 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.13 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.143 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.143 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.143 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.14 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.143 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.143 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.143 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.15 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.143 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.144 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.144 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.16 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.144 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.144 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.144 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.17 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.144 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.144 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.144 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.18 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.144 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.144 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.144 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.19 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.144 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.144 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.145 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.1a is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.145 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.145 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.145 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.1b is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.145 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.145 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.145 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.1c is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.145 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.145 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.145 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.1d is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.145 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.145 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.145 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.1e is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.146 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.146 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.146 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.1f is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-17T21:22:51.146 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.146 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.146 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:22:51.146 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.146 INFO:tasks.workunit.client.0.trial031.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-17T21:22:51.146 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:22:51.146 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:22:51.146 INFO:tasks.workunit.client.0.trial031.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-17T21:22:51.146 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:22:51.146 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.147 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:22:51.147 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.147 INFO:tasks.workunit.client.0.trial031.stdout: "message": "application not enabled on pool 'test'" 2026-02-17T21:22:51.147 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.147 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.147 INFO:tasks.workunit.client.0.trial031.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-17T21:22:51.147 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.147 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.147 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:22:51.147 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.147 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.147 INFO:tasks.workunit.client.0.trial031.stdout: "mutes": [] 2026-02-17T21:22:51.147 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.147 INFO:tasks.workunit.client.0.trial031.stdout: "monmap_first_committed": 1, 2026-02-17T21:22:51.148 INFO:tasks.workunit.client.0.trial031.stdout: "monmap_last_committed": 1, 2026-02-17T21:22:51.148 INFO:tasks.workunit.client.0.trial031.stdout: "monmap": { 2026-02-17T21:22:51.148 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 1, 2026-02-17T21:22:51.148 INFO:tasks.workunit.client.0.trial031.stdout: "fsid": "5ebac6e7-5321-4893-ad68-c06127ed455b", 2026-02-17T21:22:51.148 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:20:44.224453Z", 2026-02-17T21:22:51.148 INFO:tasks.workunit.client.0.trial031.stdout: "created": "2026-02-17T21:20:44.224453Z", 2026-02-17T21:22:51.148 INFO:tasks.workunit.client.0.trial031.stdout: "min_mon_release": 19, 2026-02-17T21:22:51.148 INFO:tasks.workunit.client.0.trial031.stdout: "min_mon_release_name": "squid", 2026-02-17T21:22:51.148 INFO:tasks.workunit.client.0.trial031.stdout: "election_strategy": 1, 2026-02-17T21:22:51.148 INFO:tasks.workunit.client.0.trial031.stdout: "disallowed_leaders": "", 2026-02-17T21:22:51.148 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode": false, 2026-02-17T21:22:51.149 INFO:tasks.workunit.client.0.trial031.stdout: "tiebreaker_mon": "", 2026-02-17T21:22:51.149 INFO:tasks.workunit.client.0.trial031.stdout: "removed_ranks": "", 2026-02-17T21:22:51.149 INFO:tasks.workunit.client.0.trial031.stdout: "features": { 2026-02-17T21:22:51.149 INFO:tasks.workunit.client.0.trial031.stdout: "persistent": [ 2026-02-17T21:22:51.149 INFO:tasks.workunit.client.0.trial031.stdout: "kraken", 2026-02-17T21:22:51.149 INFO:tasks.workunit.client.0.trial031.stdout: "luminous", 2026-02-17T21:22:51.149 INFO:tasks.workunit.client.0.trial031.stdout: "mimic", 2026-02-17T21:22:51.149 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap-prune", 2026-02-17T21:22:51.149 INFO:tasks.workunit.client.0.trial031.stdout: "nautilus", 2026-02-17T21:22:51.149 INFO:tasks.workunit.client.0.trial031.stdout: "octopus", 2026-02-17T21:22:51.149 INFO:tasks.workunit.client.0.trial031.stdout: "pacific", 2026-02-17T21:22:51.149 INFO:tasks.workunit.client.0.trial031.stdout: "elector-pinging", 2026-02-17T21:22:51.149 INFO:tasks.workunit.client.0.trial031.stdout: "quincy", 2026-02-17T21:22:51.150 INFO:tasks.workunit.client.0.trial031.stdout: "reef", 2026-02-17T21:22:51.150 INFO:tasks.workunit.client.0.trial031.stdout: "squid" 2026-02-17T21:22:51.150 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.150 INFO:tasks.workunit.client.0.trial031.stdout: "optional": [] 2026-02-17T21:22:51.150 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.150 INFO:tasks.workunit.client.0.trial031.stdout: "mons": [ 2026-02-17T21:22:51.150 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.150 INFO:tasks.workunit.client.0.trial031.stdout: "rank": 0, 2026-02-17T21:22:51.150 INFO:tasks.workunit.client.0.trial031.stdout: "name": "a", 2026-02-17T21:22:51.150 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:22:51.150 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:51.150 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.150 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:51.151 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:7115", 2026-02-17T21:22:51.151 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 0 2026-02-17T21:22:51.151 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.151 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.151 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.151 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:7115/0", 2026-02-17T21:22:51.151 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-17T21:22:51.151 INFO:tasks.workunit.client.0.trial031.stdout: "priority": 0, 2026-02-17T21:22:51.151 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 0, 2026-02-17T21:22:51.151 INFO:tasks.workunit.client.0.trial031.stdout: "crush_location": "{}" 2026-02-17T21:22:51.151 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.151 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.151 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.152 INFO:tasks.workunit.client.0.trial031.stdout: "quorum": [ 2026-02-17T21:22:51.152 INFO:tasks.workunit.client.0.trial031.stdout: 0 2026-02-17T21:22:51.152 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.152 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap": { 2026-02-17T21:22:51.152 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 45, 2026-02-17T21:22:51.152 INFO:tasks.workunit.client.0.trial031.stdout: "fsid": "5ebac6e7-5321-4893-ad68-c06127ed455b", 2026-02-17T21:22:51.152 INFO:tasks.workunit.client.0.trial031.stdout: "created": "2026-02-17T21:20:44.301417+0000", 2026-02-17T21:22:51.152 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:21:50.541606+0000", 2026-02-17T21:22:51.152 INFO:tasks.workunit.client.0.trial031.stdout: "last_up_change": "2026-02-17T21:21:49.388629+0000", 2026-02-17T21:22:51.152 INFO:tasks.workunit.client.0.trial031.stdout: "last_in_change": "2026-02-17T21:21:50.439368+0000", 2026-02-17T21:22:51.152 INFO:tasks.workunit.client.0.trial031.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-17T21:22:51.152 INFO:tasks.workunit.client.0.trial031.stdout: "flags_num": 5800384, 2026-02-17T21:22:51.153 INFO:tasks.workunit.client.0.trial031.stdout: "flags_set": [ 2026-02-17T21:22:51.153 INFO:tasks.workunit.client.0.trial031.stdout: "nodown", 2026-02-17T21:22:51.153 INFO:tasks.workunit.client.0.trial031.stdout: "noin", 2026-02-17T21:22:51.153 INFO:tasks.workunit.client.0.trial031.stdout: "noout", 2026-02-17T21:22:51.153 INFO:tasks.workunit.client.0.trial031.stdout: "pglog_hardlimit", 2026-02-17T21:22:51.153 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snapdirs", 2026-02-17T21:22:51.153 INFO:tasks.workunit.client.0.trial031.stdout: "recovery_deletes", 2026-02-17T21:22:51.153 INFO:tasks.workunit.client.0.trial031.stdout: "sortbitwise" 2026-02-17T21:22:51.153 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.153 INFO:tasks.workunit.client.0.trial031.stdout: "crush_version": 9, 2026-02-17T21:22:51.153 INFO:tasks.workunit.client.0.trial031.stdout: "full_ratio": 0.99000000953674316, 2026-02-17T21:22:51.153 INFO:tasks.workunit.client.0.trial031.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-17T21:22:51.153 INFO:tasks.workunit.client.0.trial031.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-17T21:22:51.153 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_snapshot": "", 2026-02-17T21:22:51.154 INFO:tasks.workunit.client.0.trial031.stdout: "pool_max": 1, 2026-02-17T21:22:51.154 INFO:tasks.workunit.client.0.trial031.stdout: "max_osd": 3, 2026-02-17T21:22:51.154 INFO:tasks.workunit.client.0.trial031.stdout: "require_min_compat_client": "luminous", 2026-02-17T21:22:51.154 INFO:tasks.workunit.client.0.trial031.stdout: "min_compat_client": "jewel", 2026-02-17T21:22:51.154 INFO:tasks.workunit.client.0.trial031.stdout: "require_osd_release": "squid", 2026-02-17T21:22:51.154 INFO:tasks.workunit.client.0.trial031.stdout: "allow_crimson": false, 2026-02-17T21:22:51.154 INFO:tasks.workunit.client.0.trial031.stdout: "pools": [ 2026-02-17T21:22:51.154 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.154 INFO:tasks.workunit.client.0.trial031.stdout: "pool": 1, 2026-02-17T21:22:51.154 INFO:tasks.workunit.client.0.trial031.stdout: "pool_name": "test", 2026-02-17T21:22:51.154 INFO:tasks.workunit.client.0.trial031.stdout: "create_time": "2026-02-17T21:21:13.412557+0000", 2026-02-17T21:22:51.154 INFO:tasks.workunit.client.0.trial031.stdout: "flags": 1, 2026-02-17T21:22:51.154 INFO:tasks.workunit.client.0.trial031.stdout: "flags_names": "hashpspool", 2026-02-17T21:22:51.155 INFO:tasks.workunit.client.0.trial031.stdout: "type": 1, 2026-02-17T21:22:51.155 INFO:tasks.workunit.client.0.trial031.stdout: "size": 3, 2026-02-17T21:22:51.155 INFO:tasks.workunit.client.0.trial031.stdout: "min_size": 2, 2026-02-17T21:22:51.155 INFO:tasks.workunit.client.0.trial031.stdout: "crush_rule": 0, 2026-02-17T21:22:51.155 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_count": 0, 2026-02-17T21:22:51.155 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_target": 0, 2026-02-17T21:22:51.155 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_barrier": 0, 2026-02-17T21:22:51.155 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-17T21:22:51.155 INFO:tasks.workunit.client.0.trial031.stdout: "is_stretch_pool": false, 2026-02-17T21:22:51.155 INFO:tasks.workunit.client.0.trial031.stdout: "object_hash": 2, 2026-02-17T21:22:51.155 INFO:tasks.workunit.client.0.trial031.stdout: "pg_autoscale_mode": "on", 2026-02-17T21:22:51.155 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num": 32, 2026-02-17T21:22:51.155 INFO:tasks.workunit.client.0.trial031.stdout: "pg_placement_num": 1, 2026-02-17T21:22:51.156 INFO:tasks.workunit.client.0.trial031.stdout: "pg_placement_num_target": 32, 2026-02-17T21:22:51.156 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num_target": 32, 2026-02-17T21:22:51.156 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num_pending": 32, 2026-02-17T21:22:51.156 INFO:tasks.workunit.client.0.trial031.stdout: "last_pg_merge_meta": { 2026-02-17T21:22:51.156 INFO:tasks.workunit.client.0.trial031.stdout: "source_pgid": "0.0", 2026-02-17T21:22:51.156 INFO:tasks.workunit.client.0.trial031.stdout: "ready_epoch": 0, 2026-02-17T21:22:51.156 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_started": 0, 2026-02-17T21:22:51.156 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_clean": 0, 2026-02-17T21:22:51.156 INFO:tasks.workunit.client.0.trial031.stdout: "source_version": "0'0", 2026-02-17T21:22:51.156 INFO:tasks.workunit.client.0.trial031.stdout: "target_version": "0'0" 2026-02-17T21:22:51.156 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.156 INFO:tasks.workunit.client.0.trial031.stdout: "last_change": "38", 2026-02-17T21:22:51.156 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend": "0", 2026-02-17T21:22:51.157 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-17T21:22:51.157 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend_preluminous": "38", 2026-02-17T21:22:51.157 INFO:tasks.workunit.client.0.trial031.stdout: "auid": 0, 2026-02-17T21:22:51.157 INFO:tasks.workunit.client.0.trial031.stdout: "snap_mode": "selfmanaged", 2026-02-17T21:22:51.157 INFO:tasks.workunit.client.0.trial031.stdout: "snap_seq": 0, 2026-02-17T21:22:51.157 INFO:tasks.workunit.client.0.trial031.stdout: "snap_epoch": 0, 2026-02-17T21:22:51.157 INFO:tasks.workunit.client.0.trial031.stdout: "pool_snaps": [], 2026-02-17T21:22:51.157 INFO:tasks.workunit.client.0.trial031.stdout: "removed_snaps": "[]", 2026-02-17T21:22:51.157 INFO:tasks.workunit.client.0.trial031.stdout: "quota_max_bytes": 0, 2026-02-17T21:22:51.157 INFO:tasks.workunit.client.0.trial031.stdout: "quota_max_objects": 0, 2026-02-17T21:22:51.157 INFO:tasks.workunit.client.0.trial031.stdout: "tiers": [], 2026-02-17T21:22:51.157 INFO:tasks.workunit.client.0.trial031.stdout: "tier_of": -1, 2026-02-17T21:22:51.157 INFO:tasks.workunit.client.0.trial031.stdout: "read_tier": -1, 2026-02-17T21:22:51.157 INFO:tasks.workunit.client.0.trial031.stdout: "write_tier": -1, 2026-02-17T21:22:51.158 INFO:tasks.workunit.client.0.trial031.stdout: "cache_mode": "none", 2026-02-17T21:22:51.158 INFO:tasks.workunit.client.0.trial031.stdout: "target_max_bytes": 0, 2026-02-17T21:22:51.158 INFO:tasks.workunit.client.0.trial031.stdout: "target_max_objects": 0, 2026-02-17T21:22:51.158 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-17T21:22:51.158 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-17T21:22:51.158 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-17T21:22:51.158 INFO:tasks.workunit.client.0.trial031.stdout: "cache_min_flush_age": 0, 2026-02-17T21:22:51.158 INFO:tasks.workunit.client.0.trial031.stdout: "cache_min_evict_age": 0, 2026-02-17T21:22:51.158 INFO:tasks.workunit.client.0.trial031.stdout: "erasure_code_profile": "", 2026-02-17T21:22:51.158 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_params": { 2026-02-17T21:22:51.158 INFO:tasks.workunit.client.0.trial031.stdout: "type": "none" 2026-02-17T21:22:51.158 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.158 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_period": 0, 2026-02-17T21:22:51.159 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_count": 0, 2026-02-17T21:22:51.159 INFO:tasks.workunit.client.0.trial031.stdout: "use_gmt_hitset": true, 2026-02-17T21:22:51.159 INFO:tasks.workunit.client.0.trial031.stdout: "min_read_recency_for_promote": 0, 2026-02-17T21:22:51.159 INFO:tasks.workunit.client.0.trial031.stdout: "min_write_recency_for_promote": 0, 2026-02-17T21:22:51.159 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_grade_decay_rate": 0, 2026-02-17T21:22:51.159 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_search_last_n": 0, 2026-02-17T21:22:51.159 INFO:tasks.workunit.client.0.trial031.stdout: "grade_table": [], 2026-02-17T21:22:51.159 INFO:tasks.workunit.client.0.trial031.stdout: "stripe_width": 0, 2026-02-17T21:22:51.159 INFO:tasks.workunit.client.0.trial031.stdout: "expected_num_objects": 0, 2026-02-17T21:22:51.159 INFO:tasks.workunit.client.0.trial031.stdout: "fast_read": false, 2026-02-17T21:22:51.159 INFO:tasks.workunit.client.0.trial031.stdout: "options": {}, 2026-02-17T21:22:51.159 INFO:tasks.workunit.client.0.trial031.stdout: "application_metadata": {}, 2026-02-17T21:22:51.159 INFO:tasks.workunit.client.0.trial031.stdout: "read_balance": { 2026-02-17T21:22:51.160 INFO:tasks.workunit.client.0.trial031.stdout: "score_type": "Fair distribution", 2026-02-17T21:22:51.160 INFO:tasks.workunit.client.0.trial031.stdout: "score_acting": 2.9900000095367432, 2026-02-17T21:22:51.160 INFO:tasks.workunit.client.0.trial031.stdout: "score_stable": 2.9900000095367432, 2026-02-17T21:22:51.160 INFO:tasks.workunit.client.0.trial031.stdout: "optimal_score": 0.67000001668930054, 2026-02-17T21:22:51.160 INFO:tasks.workunit.client.0.trial031.stdout: "raw_score_acting": 2, 2026-02-17T21:22:51.160 INFO:tasks.workunit.client.0.trial031.stdout: "raw_score_stable": 2, 2026-02-17T21:22:51.160 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity_weighted": 1, 2026-02-17T21:22:51.160 INFO:tasks.workunit.client.0.trial031.stdout: "average_primary_affinity": 1, 2026-02-17T21:22:51.160 INFO:tasks.workunit.client.0.trial031.stdout: "average_primary_affinity_weighted": 1 2026-02-17T21:22:51.160 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.160 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.160 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.160 INFO:tasks.workunit.client.0.trial031.stdout: "osds": [ 2026-02-17T21:22:51.161 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.161 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:22:51.161 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "c804922e-7eb2-4582-9dc2-d361d3e2c25a", 2026-02-17T21:22:51.161 INFO:tasks.workunit.client.0.trial031.stdout: "up": 1, 2026-02-17T21:22:51.161 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:22:51.161 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:22:51.161 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:22:51.161 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 5, 2026-02-17T21:22:51.161 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 28, 2026-02-17T21:22:51.161 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 35, 2026-02-17T21:22:51.161 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 44, 2026-02-17T21:22:51.161 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 29, 2026-02-17T21:22:51.161 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:22:51.161 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:22:51.162 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:51.162 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.162 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:51.162 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6802", 2026-02-17T21:22:51.162 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:51.162 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.162 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.162 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:51.162 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6803", 2026-02-17T21:22:51.162 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:51.162 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.162 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.162 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.162 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:22:51.163 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:51.163 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.163 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:51.163 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6804", 2026-02-17T21:22:51.163 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:51.163 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.163 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.163 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:51.163 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6805", 2026-02-17T21:22:51.163 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:51.163 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.163 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.163 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.164 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:22:51.164 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:51.164 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.164 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:51.164 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6808", 2026-02-17T21:22:51.164 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:51.164 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.164 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.164 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:51.164 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6809", 2026-02-17T21:22:51.164 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:51.164 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.164 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.165 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.165 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:22:51.165 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:51.165 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.165 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:51.165 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6806", 2026-02-17T21:22:51.165 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:51.165 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.165 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.165 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:51.165 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6807", 2026-02-17T21:22:51.165 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3194159082 2026-02-17T21:22:51.165 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.165 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.166 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.166 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6803/3194159082", 2026-02-17T21:22:51.166 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6805/3194159082", 2026-02-17T21:22:51.166 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6809/3194159082", 2026-02-17T21:22:51.166 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6807/3194159082", 2026-02-17T21:22:51.166 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:22:51.166 INFO:tasks.workunit.client.0.trial031.stdout: "exists", 2026-02-17T21:22:51.166 INFO:tasks.workunit.client.0.trial031.stdout: "up" 2026-02-17T21:22:51.166 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.166 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.166 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.166 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:22:51.167 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "2bdda522-e523-463c-b285-abb08be7107e", 2026-02-17T21:22:51.167 INFO:tasks.workunit.client.0.trial031.stdout: "up": 0, 2026-02-17T21:22:51.167 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:22:51.167 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:22:51.167 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:22:51.167 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 0, 2026-02-17T21:22:51.167 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 0, 2026-02-17T21:22:51.167 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 9, 2026-02-17T21:22:51.167 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 21, 2026-02-17T21:22:51.167 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 30, 2026-02-17T21:22:51.167 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:22:51.167 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:22:51.167 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:51.167 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.168 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:51.168 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6810", 2026-02-17T21:22:51.168 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:51.168 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.168 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.168 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:51.168 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6811", 2026-02-17T21:22:51.168 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:51.168 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.168 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.168 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.168 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:22:51.169 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:51.169 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.169 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:51.169 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6812", 2026-02-17T21:22:51.169 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:51.169 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.169 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.169 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:51.169 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6813", 2026-02-17T21:22:51.169 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:51.169 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.169 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.169 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.169 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:22:51.169 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:51.170 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.170 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:51.170 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6816", 2026-02-17T21:22:51.170 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:51.170 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.170 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.170 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:51.170 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6817", 2026-02-17T21:22:51.170 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:51.170 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.170 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.170 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.170 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:22:51.170 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:51.170 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.171 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:51.171 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6814", 2026-02-17T21:22:51.171 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:51.171 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.171 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.171 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:51.171 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6815", 2026-02-17T21:22:51.171 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 376030295 2026-02-17T21:22:51.171 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.171 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.171 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.171 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6811/376030295", 2026-02-17T21:22:51.171 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6813/376030295", 2026-02-17T21:22:51.171 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6817/376030295", 2026-02-17T21:22:51.171 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6815/376030295", 2026-02-17T21:22:51.172 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:22:51.172 INFO:tasks.workunit.client.0.trial031.stdout: "exists" 2026-02-17T21:22:51.172 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.172 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.172 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.172 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:22:51.172 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "258ca9b7-06c7-4314-ac32-bb7e4d29e018", 2026-02-17T21:22:51.172 INFO:tasks.workunit.client.0.trial031.stdout: "up": 1, 2026-02-17T21:22:51.172 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:22:51.172 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:22:51.172 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:22:51.172 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 13, 2026-02-17T21:22:51.172 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 28, 2026-02-17T21:22:51.172 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 40, 2026-02-17T21:22:51.172 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 0, 2026-02-17T21:22:51.173 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 31, 2026-02-17T21:22:51.173 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:22:51.173 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:22:51.173 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:51.173 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.173 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:51.173 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6810", 2026-02-17T21:22:51.173 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:51.173 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.173 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.173 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:51.173 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6811", 2026-02-17T21:22:51.173 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:51.173 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.174 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.174 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.174 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:22:51.174 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:51.174 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.174 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:51.174 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6812", 2026-02-17T21:22:51.174 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:51.174 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.174 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.174 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:51.174 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6813", 2026-02-17T21:22:51.174 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:51.174 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.174 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.175 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.175 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:22:51.175 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:51.175 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.175 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:51.175 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6816", 2026-02-17T21:22:51.175 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:51.175 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.175 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.175 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:51.175 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6817", 2026-02-17T21:22:51.175 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:51.175 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.175 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6814", 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6815", 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 1416133595 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6811/1416133595", 2026-02-17T21:22:51.176 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6813/1416133595", 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6817/1416133595", 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6815/1416133595", 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: "exists", 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: "up" 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: "osd_xinfo": [ 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:21:38.109278+0000", 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:22:51.177 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "2026-02-17T21:20:53.042100+0000", 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:21:38.381347+0000", 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:21:38.685698+0000", 2026-02-17T21:22:51.178 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "2026-02-17T21:21:11.695133+0000", 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap": [], 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap_items": [], 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap_primaries": [], 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "pg_temp": [], 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "primary_temp": [], 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "blocklist": {}, 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "range_blocklist": {}, 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "erasure_code_profiles": { 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "default": { 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "k": "2", 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "m": "2", 2026-02-17T21:22:51.179 INFO:tasks.workunit.client.0.trial031.stdout: "plugin": "jerasure", 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: "technique": "reed_sol_van" 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: "removed_snaps_queue": [], 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: "new_removed_snaps": [], 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: "new_purged_snaps": [], 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: "crush_node_flags": {}, 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: "device_class_flags": {}, 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode": { 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode_enabled": false, 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_bucket_count": 0, 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: "degraded_stretch_mode": 0, 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: "recovering_stretch_mode": 0, 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode_bucket": 0 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: "osd_metadata": [ 2026-02-17T21:22:51.180 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6804/3194159082,v1:127.0.0.1:6805/3194159082]", 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-17T21:22:51.181 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:20:46.458290Z", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6802/3194159082,v1:127.0.0.1:6803/3194159082]", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:22:51.182 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/3194159082,v1:127.0.0.1:6809/3194159082]", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/3194159082,v1:127.0.0.1:6807/3194159082]", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/0", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:22:51.183 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6812/376030295,v1:127.0.0.1:6813/376030295]", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:22:51.184 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:20:55.680906Z", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6810/376030295,v1:127.0.0.1:6811/376030295]", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/376030295,v1:127.0.0.1:6817/376030295]", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/376030295,v1:127.0.0.1:6815/376030295]", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:22:51.185 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/1", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6812/1416133595,v1:127.0.0.1:6813/1416133595]", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:22:51.186 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:21:05.125195Z", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:22:51.187 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6810/1416133595,v1:127.0.0.1:6811/1416133595]", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/1416133595,v1:127.0.0.1:6817/1416133595]", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/1416133595,v1:127.0.0.1:6815/1416133595]", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/2", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_clean_epochs": { 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "min_last_epoch_clean": 0, 2026-02-17T21:22:51.188 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_clean": { 2026-02-17T21:22:51.189 INFO:tasks.workunit.client.0.trial031.stdout: "per_pool": [] 2026-02-17T21:22:51.189 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.189 INFO:tasks.workunit.client.0.trial031.stdout: "osd_epochs": [ 2026-02-17T21:22:51.189 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.189 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:22:51.189 INFO:tasks.workunit.client.0.trial031.stderr:report 164826893 2026-02-17T21:22:51.189 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 35 2026-02-17T21:22:51.189 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.189 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.189 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:22:51.189 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 42 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_first_committed": 1, 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_last_committed": 45, 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: "crushmap": { 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: "devices": [ 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.0", 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.1", 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.2", 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.190 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: "types": [ 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 0, 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd" 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: "name": "host" 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 2, 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: "name": "chassis" 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 3, 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: "name": "rack" 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 4, 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: "name": "row" 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.191 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 5, 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: "name": "pdu" 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 6, 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: "name": "pod" 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 7, 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: "name": "room" 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 8, 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: "name": "datacenter" 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 9, 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: "name": "zone" 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 10, 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: "name": "region" 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:22:51.192 INFO:tasks.workunit.client.0.trial031.stdout: "name": "root" 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "buckets": [ 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "id": -1, 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "name": "default", 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "root", 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "id": -3, 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "id": -2, 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "name": "default~ssd", 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "root", 2026-02-17T21:22:51.193 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "id": -4, 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "id": -3, 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "name": "trial031", 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "host", 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.194 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 1 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 2 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "id": -4, 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "name": "trial031~ssd", 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "host", 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.195 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 1 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 2 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "rules": [ 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "rule_id": 0, 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "rule_name": "replicated_rule", 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "type": 1, 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "steps": [ 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "op": "take", 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "item": -1, 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "item_name": "default" 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "op": "choose_firstn", 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "num": 0, 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: "type": "osd" 2026-02-17T21:22:51.196 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "op": "emit" 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "tunables": { 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "choose_local_tries": 0, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "choose_local_fallback_tries": 0, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "choose_total_tries": 50, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_descend_once": 1, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_vary_r": 1, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_stable": 1, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "msr_descents": 100, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "msr_collision_tries": 100, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "straw_calc_version": 1, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "allowed_bucket_algs": 54, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "profile": "jewel", 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "optimal_tunables": 1, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "legacy_tunables": 0, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "minimum_required_version": "jewel", 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables": 1, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables2": 1, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "has_v2_rules": 0, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables3": 1, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "has_v3_rules": 0, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "has_v4_buckets": 1, 2026-02-17T21:22:51.197 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables5": 1, 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "has_v5_rules": 0, 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "has_msr_rules": 0 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "choose_args": {} 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "fsmap": { 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 1, 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "btime": "2026-02-17T21:20:44:301188+0000", 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "default_fscid": -1, 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "compat": { 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "compat": {}, 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "ro_compat": {}, 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "incompat": { 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "feature_1": "base v0.20", 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "feature_2": "client writeable ranges", 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "feature_3": "default file layouts on dirs", 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "feature_4": "dir inode in separate object", 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "feature_5": "mds uses versioned encoding", 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "feature_8": "no anchor table", 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "feature_9": "file layout v2", 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "feature_10": "snaprealm v2", 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "feature_11": "minor log segments", 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "feature_12": "quiesce subvolumes" 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "feature_flags": { 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "enable_multiple": true, 2026-02-17T21:22:51.198 INFO:tasks.workunit.client.0.trial031.stdout: "ever_enabled_multiple": true 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "standbys": [], 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "filesystems": [] 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "mdsmap_first_committed": 1, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "mdsmap_last_committed": 1, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "auth": { 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "first_committed": 1, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "last_committed": 5, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_secrets": 3 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg": 32, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_active": 32, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_unknown": 0, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_osd": 3, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "pool_sum": { 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "stat_sum": { 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes": 70861, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects": 100, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_clones": 0, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_copies": 300, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing": 0, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_degraded": 100, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_misplaced": 0, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_unfound": 0, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_dirty": 100, 2026-02-17T21:22:51.199 INFO:tasks.workunit.client.0.trial031.stdout: "num_whiteouts": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_read": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_read_kb": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_write": 101, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_write_kb": 101, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_scrub_errors": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_deep_scrub_errors": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_recovered": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_recovered": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_keys_recovered": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_omap": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_kb": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_kb": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_promote": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_high": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_low": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_some": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_full": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_pinned": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_legacy_snapsets": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_large_omap_objects": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_manifest": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_bytes": 0, 2026-02-17T21:22:51.200 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_keys": 0, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_repaired": 0 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "store_stats": { 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "total": 0, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "available": 0, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 0, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 0, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 0, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 0 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "log_size": 513, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "ondisk_log_size": 513, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "up": 64, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "acting": 64, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "num_store_stats": 0 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "osd_sum": { 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 0, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 0, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "num_pgs": 65, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "num_osds": 3, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_osds": 3, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "kb": 314572800, 2026-02-17T21:22:51.201 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used": 83992, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_data": 2160, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_omap": 10, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_meta": 81717, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "kb_avail": 314488808, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "statfs": { 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "total": 322122547200, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "available": 322036539392, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 2211840, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 520178, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 10389, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 83679083 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "hb_peers": [], 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "snap_trim_queue_len": 0, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "num_snap_trimming": 0, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "num_shards_repaired": 0, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "op_queue_age_hist": { 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "histogram": [], 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "upper_bound": 1 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "perf_stat": { 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ms": 0, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ms": 0, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ns": 0, 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ns": 0 2026-02-17T21:22:51.202 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "alerts": [], 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "network_ping_times": [] 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "osd_sum_by_class": { 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "ssd": { 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 0, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 0, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "num_pgs": 65, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "num_osds": 3, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_osds": 3, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "kb": 314572800, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used": 83992, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_data": 2160, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_omap": 10, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_meta": 81717, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "kb_avail": 314488808, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "statfs": { 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "total": 322122547200, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "available": 322036539392, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 2211840, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 520178, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 10389, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 83679083 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.203 INFO:tasks.workunit.client.0.trial031.stdout: "hb_peers": [], 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "snap_trim_queue_len": 0, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "num_snap_trimming": 0, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "num_shards_repaired": 0, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "op_queue_age_hist": { 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "histogram": [], 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "upper_bound": 1 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "perf_stat": { 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ms": 0, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ms": 0, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ns": 0, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ns": 0 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "alerts": [], 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "network_ping_times": [] 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "pool_stats": [ 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "poolid": 1, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg": 32, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "stat_sum": { 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes": 70861, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects": 100, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_clones": 0, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_copies": 300, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing": 0, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_degraded": 100, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_misplaced": 0, 2026-02-17T21:22:51.204 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_unfound": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_dirty": 100, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_whiteouts": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_read": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_read_kb": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_write": 101, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_write_kb": 101, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_scrub_errors": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_deep_scrub_errors": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_recovered": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_recovered": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_keys_recovered": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_omap": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_kb": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_kb": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_promote": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_high": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_low": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_some": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_full": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_pinned": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_legacy_snapsets": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_large_omap_objects": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_manifest": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_bytes": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_keys": 0, 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_repaired": 0 2026-02-17T21:22:51.205 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "store_stats": { 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "total": 0, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "available": 0, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 1220608, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 211432, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 0, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 0 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "log_size": 513, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "ondisk_log_size": 513, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "up": 64, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "acting": 64, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "num_store_stats": 3 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "osd_stats": [ 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 150323855378 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 38654705678 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 171798691857 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.206 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_by_state": [ 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "state": "active+undersized+degraded", 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "num": 32 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_by_osd": [ 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 32, 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 32, 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 32, 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 0, 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snaps": [ 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "pool": 1, 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snaps": {} 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "servicemap": { 2026-02-17T21:22:51.207 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 6, 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:22:50.769376+0000", 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stdout: "services": {} 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stdout: "mgrstat_first_committed": 1, 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stdout: "mgrstat_last_committed": 61, 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stdout: "logm_first_committed": 1, 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stdout: "logm_last_committed": 305, 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stdout: "paxos": { 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stdout: "first_committed": 1, 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stdout: "last_committed": 420, 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stdout: "last_pn": 0, 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stdout: "accepted_pn": 0 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stdout:delay recovery 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:761: TEST_divergent_3: echo 'delay recovery' 2026-02-17T21:22:51.208 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:762: TEST_divergent_3: ceph pg dump pgs 2026-02-17T21:22:51.337 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.1f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343954+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796405+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.1e 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341401+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796444+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.1d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341205+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796363+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341470+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796285+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341090+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796174+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.9 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341294+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796222+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.8 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.340967+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796205+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.7 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344971+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796303+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.6 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343790+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796248+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.5 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343964+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796257+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.4 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343986+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796276+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.3 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343765+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796266+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.2 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343766+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796231+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343947+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796293+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.1 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341729+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796196+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.1c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341106+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796421+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.0 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341066+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796213+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341537+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796240+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.e 3 0 3 0 0 2467 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:22:06.343911+0000 45'101 45:142 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:23.047342+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344345+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796355+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.10 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341474+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796388+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.11 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342237+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796312+0000 0 0 2026-02-17T21:22:51.338 INFO:tasks.workunit.client.0.trial031.stdout:1.12 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341306+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796337+0000 0 0 2026-02-17T21:22:51.339 INFO:tasks.workunit.client.0.trial031.stdout:1.13 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341120+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796429+0000 0 0 2026-02-17T21:22:51.339 INFO:tasks.workunit.client.0.trial031.stdout:1.14 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341229+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796436+0000 0 0 2026-02-17T21:22:51.339 INFO:tasks.workunit.client.0.trial031.stdout:1.15 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341182+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796346+0000 0 0 2026-02-17T21:22:51.339 INFO:tasks.workunit.client.0.trial031.stdout:1.16 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341319+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796453+0000 0 0 2026-02-17T21:22:51.339 INFO:tasks.workunit.client.0.trial031.stdout:1.17 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343856+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796413+0000 0 0 2026-02-17T21:22:51.339 INFO:tasks.workunit.client.0.trial031.stdout:1.18 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342096+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796328+0000 0 0 2026-02-17T21:22:51.339 INFO:tasks.workunit.client.0.trial031.stdout:1.19 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343794+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796380+0000 0 0 2026-02-17T21:22:51.339 INFO:tasks.workunit.client.0.trial031.stdout:1.1a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342036+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796321+0000 0 0 2026-02-17T21:22:51.339 INFO:tasks.workunit.client.0.trial031.stdout:1.1b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341167+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796371+0000 0 0 2026-02-17T21:22:51.339 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:22:51.339 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:22:51.339 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:22:51.346 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:763: TEST_divergent_3: for i in $non_divergent 2026-02-17T21:22:51.346 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: get_asok_path osd.0 2026-02-17T21:22:51.346 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T21:22:51.347 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T21:22:51.347 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:22:51.347 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:22:51.347 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:22:51.348 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.0.asok 2026-02-17T21:22:51.348 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: CEPH_ARGS= 2026-02-17T21:22:51.348 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.0.asok set_recovery_delay 100000 2026-02-17T21:22:51.413 INFO:tasks.workunit.client.0.trial031.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-17T21:22:51.419 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:763: TEST_divergent_3: for i in $non_divergent 2026-02-17T21:22:51.420 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: get_asok_path osd.2 2026-02-17T21:22:51.420 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-17T21:22:51.420 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-17T21:22:51.421 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:22:51.421 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:22:51.421 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:22:51.421 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.2.asok 2026-02-17T21:22:51.421 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: CEPH_ARGS= 2026-02-17T21:22:51.421 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.2.asok set_recovery_delay 100000 2026-02-17T21:22:51.473 INFO:tasks.workunit.client.0.trial031.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-17T21:22:51.479 INFO:tasks.workunit.client.0.trial031.stdout:revive divergent 1 2026-02-17T21:22:51.480 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:769: TEST_divergent_3: echo 'revive divergent 1' 2026-02-17T21:22:51.480 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:770: TEST_divergent_3: ceph pg dump pgs 2026-02-17T21:22:51.683 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:22:51.683 INFO:tasks.workunit.client.0.trial031.stdout:1.1f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343954+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796405+0000 0 0 2026-02-17T21:22:51.683 INFO:tasks.workunit.client.0.trial031.stdout:1.1e 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341401+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796444+0000 0 0 2026-02-17T21:22:51.683 INFO:tasks.workunit.client.0.trial031.stdout:1.1d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341205+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796363+0000 0 0 2026-02-17T21:22:51.684 INFO:tasks.workunit.client.0.trial031.stdout:1.b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341470+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796285+0000 0 0 2026-02-17T21:22:51.684 INFO:tasks.workunit.client.0.trial031.stdout:1.a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341090+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796174+0000 0 0 2026-02-17T21:22:51.684 INFO:tasks.workunit.client.0.trial031.stdout:1.9 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341294+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796222+0000 0 0 2026-02-17T21:22:51.684 INFO:tasks.workunit.client.0.trial031.stdout:1.8 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.340967+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796205+0000 0 0 2026-02-17T21:22:51.684 INFO:tasks.workunit.client.0.trial031.stdout:1.7 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344971+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796303+0000 0 0 2026-02-17T21:22:51.684 INFO:tasks.workunit.client.0.trial031.stdout:1.6 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343790+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796248+0000 0 0 2026-02-17T21:22:51.684 INFO:tasks.workunit.client.0.trial031.stdout:1.5 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343964+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796257+0000 0 0 2026-02-17T21:22:51.684 INFO:tasks.workunit.client.0.trial031.stdout:1.4 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343986+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796276+0000 0 0 2026-02-17T21:22:51.684 INFO:tasks.workunit.client.0.trial031.stdout:1.3 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343765+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796266+0000 0 0 2026-02-17T21:22:51.684 INFO:tasks.workunit.client.0.trial031.stdout:1.2 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343766+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796231+0000 0 0 2026-02-17T21:22:51.684 INFO:tasks.workunit.client.0.trial031.stdout:1.c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343947+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796293+0000 0 0 2026-02-17T21:22:51.684 INFO:tasks.workunit.client.0.trial031.stdout:1.1 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341729+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796196+0000 0 0 2026-02-17T21:22:51.685 INFO:tasks.workunit.client.0.trial031.stdout:1.1c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341106+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796421+0000 0 0 2026-02-17T21:22:51.685 INFO:tasks.workunit.client.0.trial031.stdout:1.0 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341066+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796213+0000 0 0 2026-02-17T21:22:51.685 INFO:tasks.workunit.client.0.trial031.stdout:1.d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341537+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796240+0000 0 0 2026-02-17T21:22:51.685 INFO:tasks.workunit.client.0.trial031.stdout:1.e 3 0 3 0 0 2467 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:22:06.343911+0000 45'101 45:142 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:23.047342+0000 0 0 2026-02-17T21:22:51.685 INFO:tasks.workunit.client.0.trial031.stdout:1.f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344345+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796355+0000 0 0 2026-02-17T21:22:51.685 INFO:tasks.workunit.client.0.trial031.stdout:1.10 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341474+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796388+0000 0 0 2026-02-17T21:22:51.685 INFO:tasks.workunit.client.0.trial031.stdout:1.11 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342237+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796312+0000 0 0 2026-02-17T21:22:51.685 INFO:tasks.workunit.client.0.trial031.stdout:1.12 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341306+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796337+0000 0 0 2026-02-17T21:22:51.685 INFO:tasks.workunit.client.0.trial031.stdout:1.13 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341120+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796429+0000 0 0 2026-02-17T21:22:51.685 INFO:tasks.workunit.client.0.trial031.stdout:1.14 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341229+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796436+0000 0 0 2026-02-17T21:22:51.685 INFO:tasks.workunit.client.0.trial031.stdout:1.15 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341182+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796346+0000 0 0 2026-02-17T21:22:51.685 INFO:tasks.workunit.client.0.trial031.stdout:1.16 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341319+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796453+0000 0 0 2026-02-17T21:22:51.686 INFO:tasks.workunit.client.0.trial031.stdout:1.17 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343856+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796413+0000 0 0 2026-02-17T21:22:51.686 INFO:tasks.workunit.client.0.trial031.stdout:1.18 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342096+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796328+0000 0 0 2026-02-17T21:22:51.686 INFO:tasks.workunit.client.0.trial031.stdout:1.19 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343794+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796380+0000 0 0 2026-02-17T21:22:51.686 INFO:tasks.workunit.client.0.trial031.stdout:1.1a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342036+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796321+0000 0 0 2026-02-17T21:22:51.686 INFO:tasks.workunit.client.0.trial031.stdout:1.1b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341167+0000 28'100 45:141 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:07.796371+0000 0 0 2026-02-17T21:22:51.686 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:22:51.686 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:22:51.686 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:22:51.692 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:771: TEST_divergent_3: ceph osd set noup 2026-02-17T21:22:51.995 INFO:tasks.workunit.client.0.trial031.stderr:noup is set 2026-02-17T21:22:52.008 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:772: TEST_divergent_3: activate_osd td/divergent-priors 1 2026-02-17T21:22:52.008 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:22:52.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:22:52.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T21:22:52.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:22:52.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-17T21:22:52.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:22:52.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:22:52.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:22:52.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:22:52.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-17T21:22:52.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-17T21:22:52.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:22:52.010 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:22:52.010 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:22:52.010 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:22:52.010 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:22:52.010 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:22:52.010 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:22:52.010 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:22:52.010 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:22:52.010 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:22:52.012 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:22:52.012 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:22:52.012 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:22:52.012 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:22:52.012 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:22:52.012 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:22:52.012 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:22:52.012 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:22:52.012 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:22:52.012 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:22:52.012 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-17T21:22:52.013 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T21:22:52.013 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:22:52.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:22:52.014 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-17T21:22:52.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T21:22:52.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:22:52.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:22:52.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:22:52.038 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:22:52.033+0000 7f9d6ba558c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:22:52.039 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:22:52.033+0000 7f9d6ba558c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:22:52.040 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:22:52.037+0000 7f9d6ba558c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:22:52.216 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:880: activate_osd: return 0 2026-02-17T21:22:52.216 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:773: TEST_divergent_3: sleep 5 2026-02-17T21:22:53.270 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:22:53.265+0000 7f9d6ba558c0 -1 Falling back to public interface 2026-02-17T21:22:56.297 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:22:56.292+0000 7f9d6ba558c0 -1 osd.1 28 log_to_monitors true 2026-02-17T21:22:57.218 INFO:tasks.workunit.client.0.trial031.stdout:delay recovery divergent 2026-02-17T21:22:57.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:775: TEST_divergent_3: echo 'delay recovery divergent' 2026-02-17T21:22:57.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:776: TEST_divergent_3: ceph pg dump pgs 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.1f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343954+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983452+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.1e 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341401+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983484+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.1d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341205+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983419+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341470+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983361+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341090+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983286+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.9 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341294+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983315+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.8 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.340967+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983302+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.7 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344971+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983373+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.6 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343790+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983335+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.5 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343964+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983341+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.4 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343986+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983354+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.3 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343765+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983348+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.2 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343766+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983322+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343947+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983366+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.1 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341729+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983295+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.1c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341106+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983465+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.0 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341066+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983309+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341537+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983329+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.e 3 0 3 0 0 2467 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:22:06.343911+0000 45'101 47:146 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983446+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344345+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983413+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.10 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341474+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983439+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.11 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342237+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983379+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.12 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341306+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983400+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.13 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341120+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983471+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.14 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341229+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983477+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.15 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341182+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983406+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.16 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341319+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983490+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.17 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343856+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983458+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.18 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342096+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983393+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.19 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343794+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983433+0000 0 0 2026-02-17T21:22:57.404 INFO:tasks.workunit.client.0.trial031.stdout:1.1a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342036+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983386+0000 0 0 2026-02-17T21:22:57.405 INFO:tasks.workunit.client.0.trial031.stdout:1.1b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341167+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983426+0000 0 0 2026-02-17T21:22:57.405 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:22:57.405 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:22:57.405 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:22:57.412 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:777: TEST_divergent_3: get_asok_path osd.1 2026-02-17T21:22:57.412 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T21:22:57.412 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T21:22:57.412 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:22:57.412 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:22:57.412 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:22:57.413 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.1.asok 2026-02-17T21:22:57.413 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:777: TEST_divergent_3: CEPH_ARGS= 2026-02-17T21:22:57.413 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:777: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.1.asok set_recovery_delay 100000 2026-02-17T21:22:57.467 INFO:tasks.workunit.client.0.trial031.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-17T21:22:57.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:779: TEST_divergent_3: ceph osd unset noup 2026-02-17T21:22:57.780 INFO:tasks.workunit.client.0.trial031.stderr:noup is unset 2026-02-17T21:22:57.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:781: TEST_divergent_3: wait_for_osd up 0 2026-02-17T21:22:57.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:22:57.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:22:57.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:22:57.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:22:57.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:22:57.793 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:22:57.793 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:22:57.793 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:22:57.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:22:58.001 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 35 up_thru 44 down_at 29 last_clean_interval [5,28) [v2:127.0.0.1:6802/3194159082,v1:127.0.0.1:6803/3194159082] [v2:127.0.0.1:6804/3194159082,v1:127.0.0.1:6805/3194159082] exists,up c804922e-7eb2-4582-9dc2-d361d3e2c25a 2026-02-17T21:22:58.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:22:58.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:22:58.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:22:58.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:782: TEST_divergent_3: wait_for_osd up 1 2026-02-17T21:22:58.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:22:58.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:22:58.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:22:58.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:22:58.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:22:58.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:22:58.003 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:22:58.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:22:58.004 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:22:58.204 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 50 up_thru 50 down_at 30 last_clean_interval [9,28) [v2:127.0.0.1:6818/3457322548,v1:127.0.0.1:6819/3457322548] [v2:127.0.0.1:6820/3457322548,v1:127.0.0.1:6821/3457322548] exists,up 2bdda522-e523-463c-b285-abb08be7107e 2026-02-17T21:22:58.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:22:58.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:22:58.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:22:58.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:783: TEST_divergent_3: wait_for_osd up 2 2026-02-17T21:22:58.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:22:58.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:22:58.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:22:58.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:22:58.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:22:58.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:22:58.207 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:22:58.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:22:58.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:22:58.401 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 40 up_thru 0 down_at 31 last_clean_interval [13,28) [v2:127.0.0.1:6810/1416133595,v1:127.0.0.1:6811/1416133595] [v2:127.0.0.1:6812/1416133595,v1:127.0.0.1:6813/1416133595] exists,up 258ca9b7-06c7-4314-ac32-bb7e4d29e018 2026-02-17T21:22:58.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:22:58.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:22:58.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:22:58.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:785: TEST_divergent_3: ceph pg dump pgs 2026-02-17T21:22:58.587 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:22:58.587 INFO:tasks.workunit.client.0.trial031.stdout:1.1f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343954+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983452+0000 0 0 2026-02-17T21:22:58.587 INFO:tasks.workunit.client.0.trial031.stdout:1.1e 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341401+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983484+0000 0 0 2026-02-17T21:22:58.587 INFO:tasks.workunit.client.0.trial031.stdout:1.1d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341205+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983419+0000 0 0 2026-02-17T21:22:58.587 INFO:tasks.workunit.client.0.trial031.stdout:1.b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341470+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983361+0000 0 0 2026-02-17T21:22:58.588 INFO:tasks.workunit.client.0.trial031.stdout:1.a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341090+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983286+0000 0 0 2026-02-17T21:22:58.588 INFO:tasks.workunit.client.0.trial031.stdout:1.9 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341294+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983315+0000 0 0 2026-02-17T21:22:58.588 INFO:tasks.workunit.client.0.trial031.stdout:1.8 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.340967+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983302+0000 0 0 2026-02-17T21:22:58.588 INFO:tasks.workunit.client.0.trial031.stdout:1.7 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344971+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983373+0000 0 0 2026-02-17T21:22:58.588 INFO:tasks.workunit.client.0.trial031.stdout:1.6 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343790+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983335+0000 0 0 2026-02-17T21:22:58.588 INFO:tasks.workunit.client.0.trial031.stdout:1.5 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343964+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983341+0000 0 0 2026-02-17T21:22:58.588 INFO:tasks.workunit.client.0.trial031.stdout:1.4 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343986+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983354+0000 0 0 2026-02-17T21:22:58.588 INFO:tasks.workunit.client.0.trial031.stdout:1.3 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343765+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983348+0000 0 0 2026-02-17T21:22:58.588 INFO:tasks.workunit.client.0.trial031.stdout:1.2 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343766+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983322+0000 0 0 2026-02-17T21:22:58.588 INFO:tasks.workunit.client.0.trial031.stdout:1.c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343947+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983366+0000 0 0 2026-02-17T21:22:58.589 INFO:tasks.workunit.client.0.trial031.stdout:1.1 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341729+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983295+0000 0 0 2026-02-17T21:22:58.589 INFO:tasks.workunit.client.0.trial031.stdout:1.1c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341106+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983465+0000 0 0 2026-02-17T21:22:58.589 INFO:tasks.workunit.client.0.trial031.stdout:1.0 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341066+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983309+0000 0 0 2026-02-17T21:22:58.589 INFO:tasks.workunit.client.0.trial031.stdout:1.d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341537+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983329+0000 0 0 2026-02-17T21:22:58.589 INFO:tasks.workunit.client.0.trial031.stdout:1.e 3 0 3 0 0 2467 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:22:06.343911+0000 45'101 47:146 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983446+0000 0 0 2026-02-17T21:22:58.589 INFO:tasks.workunit.client.0.trial031.stdout:1.f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344345+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983413+0000 0 0 2026-02-17T21:22:58.589 INFO:tasks.workunit.client.0.trial031.stdout:1.10 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341474+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983439+0000 0 0 2026-02-17T21:22:58.589 INFO:tasks.workunit.client.0.trial031.stdout:1.11 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342237+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983379+0000 0 0 2026-02-17T21:22:58.589 INFO:tasks.workunit.client.0.trial031.stdout:1.12 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341306+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983400+0000 0 0 2026-02-17T21:22:58.589 INFO:tasks.workunit.client.0.trial031.stdout:1.13 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341120+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983471+0000 0 0 2026-02-17T21:22:58.589 INFO:tasks.workunit.client.0.trial031.stdout:1.14 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341229+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983477+0000 0 0 2026-02-17T21:22:58.589 INFO:tasks.workunit.client.0.trial031.stdout:1.15 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341182+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983406+0000 0 0 2026-02-17T21:22:58.590 INFO:tasks.workunit.client.0.trial031.stdout:1.16 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341319+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983490+0000 0 0 2026-02-17T21:22:58.590 INFO:tasks.workunit.client.0.trial031.stdout:1.17 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343856+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983458+0000 0 0 2026-02-17T21:22:58.590 INFO:tasks.workunit.client.0.trial031.stdout:1.18 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342096+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983393+0000 0 0 2026-02-17T21:22:58.590 INFO:tasks.workunit.client.0.trial031.stdout:1.19 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343794+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983433+0000 0 0 2026-02-17T21:22:58.590 INFO:tasks.workunit.client.0.trial031.stdout:1.1a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342036+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983386+0000 0 0 2026-02-17T21:22:58.590 INFO:tasks.workunit.client.0.trial031.stdout:1.1b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341167+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983426+0000 0 0 2026-02-17T21:22:58.590 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:22:58.590 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:22:58.590 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:22:58.595 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:786: TEST_divergent_3: echo 'wait for peering' 2026-02-17T21:22:58.595 INFO:tasks.workunit.client.0.trial031.stdout:wait for peering 2026-02-17T21:22:58.596 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:787: TEST_divergent_3: ceph pg dump pgs 2026-02-17T21:22:58.797 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:22:58.798 INFO:tasks.workunit.client.0.trial031.stdout:1.1f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343954+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983452+0000 0 0 2026-02-17T21:22:58.798 INFO:tasks.workunit.client.0.trial031.stdout:1.1e 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341401+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983484+0000 0 0 2026-02-17T21:22:58.798 INFO:tasks.workunit.client.0.trial031.stdout:1.1d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341205+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983419+0000 0 0 2026-02-17T21:22:58.798 INFO:tasks.workunit.client.0.trial031.stdout:1.b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341470+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983361+0000 0 0 2026-02-17T21:22:58.798 INFO:tasks.workunit.client.0.trial031.stdout:1.a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341090+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983286+0000 0 0 2026-02-17T21:22:58.798 INFO:tasks.workunit.client.0.trial031.stdout:1.9 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341294+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983315+0000 0 0 2026-02-17T21:22:58.798 INFO:tasks.workunit.client.0.trial031.stdout:1.8 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.340967+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983302+0000 0 0 2026-02-17T21:22:58.798 INFO:tasks.workunit.client.0.trial031.stdout:1.7 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344971+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983373+0000 0 0 2026-02-17T21:22:58.798 INFO:tasks.workunit.client.0.trial031.stdout:1.6 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343790+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983335+0000 0 0 2026-02-17T21:22:58.798 INFO:tasks.workunit.client.0.trial031.stdout:1.5 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343964+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983341+0000 0 0 2026-02-17T21:22:58.799 INFO:tasks.workunit.client.0.trial031.stdout:1.4 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343986+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983354+0000 0 0 2026-02-17T21:22:58.799 INFO:tasks.workunit.client.0.trial031.stdout:1.3 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343765+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983348+0000 0 0 2026-02-17T21:22:58.799 INFO:tasks.workunit.client.0.trial031.stdout:1.2 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343766+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983322+0000 0 0 2026-02-17T21:22:58.799 INFO:tasks.workunit.client.0.trial031.stdout:1.c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343947+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983366+0000 0 0 2026-02-17T21:22:58.799 INFO:tasks.workunit.client.0.trial031.stdout:1.1 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341729+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983295+0000 0 0 2026-02-17T21:22:58.799 INFO:tasks.workunit.client.0.trial031.stdout:1.1c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341106+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983465+0000 0 0 2026-02-17T21:22:58.799 INFO:tasks.workunit.client.0.trial031.stdout:1.0 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341066+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983309+0000 0 0 2026-02-17T21:22:58.799 INFO:tasks.workunit.client.0.trial031.stdout:1.d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341537+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983329+0000 0 0 2026-02-17T21:22:58.799 INFO:tasks.workunit.client.0.trial031.stdout:1.e 3 0 3 0 0 2467 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:22:06.343911+0000 45'101 47:146 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983446+0000 0 0 2026-02-17T21:22:58.799 INFO:tasks.workunit.client.0.trial031.stdout:1.f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344345+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983413+0000 0 0 2026-02-17T21:22:58.799 INFO:tasks.workunit.client.0.trial031.stdout:1.10 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341474+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983439+0000 0 0 2026-02-17T21:22:58.799 INFO:tasks.workunit.client.0.trial031.stdout:1.11 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342237+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983379+0000 0 0 2026-02-17T21:22:58.800 INFO:tasks.workunit.client.0.trial031.stdout:1.12 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341306+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983400+0000 0 0 2026-02-17T21:22:58.800 INFO:tasks.workunit.client.0.trial031.stdout:1.13 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341120+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983471+0000 0 0 2026-02-17T21:22:58.800 INFO:tasks.workunit.client.0.trial031.stdout:1.14 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341229+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983477+0000 0 0 2026-02-17T21:22:58.800 INFO:tasks.workunit.client.0.trial031.stdout:1.15 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341182+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983406+0000 0 0 2026-02-17T21:22:58.800 INFO:tasks.workunit.client.0.trial031.stdout:1.16 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341319+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983490+0000 0 0 2026-02-17T21:22:58.800 INFO:tasks.workunit.client.0.trial031.stdout:1.17 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343856+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983458+0000 0 0 2026-02-17T21:22:58.800 INFO:tasks.workunit.client.0.trial031.stdout:1.18 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342096+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983393+0000 0 0 2026-02-17T21:22:58.800 INFO:tasks.workunit.client.0.trial031.stdout:1.19 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343794+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983433+0000 0 0 2026-02-17T21:22:58.800 INFO:tasks.workunit.client.0.trial031.stdout:1.1a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342036+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983386+0000 0 0 2026-02-17T21:22:58.800 INFO:tasks.workunit.client.0.trial031.stdout:1.1b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341167+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983426+0000 0 0 2026-02-17T21:22:58.800 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:22:58.801 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:22:58.801 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:22:58.806 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:788: TEST_divergent_3: rados -p test put foo /etc/fstab 2026-02-17T21:22:58.837 INFO:tasks.workunit.client.0.trial031.stdout:killing divergent 1 2026-02-17T21:22:58.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:792: TEST_divergent_3: echo 'killing divergent 1' 2026-02-17T21:22:58.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:793: TEST_divergent_3: ceph pg dump pgs 2026-02-17T21:22:59.027 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:22:59.028 INFO:tasks.workunit.client.0.trial031.stdout:1.1f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343954+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983452+0000 0 0 2026-02-17T21:22:59.028 INFO:tasks.workunit.client.0.trial031.stdout:1.1e 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341401+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983484+0000 0 0 2026-02-17T21:22:59.028 INFO:tasks.workunit.client.0.trial031.stdout:1.1d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341205+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983419+0000 0 0 2026-02-17T21:22:59.028 INFO:tasks.workunit.client.0.trial031.stdout:1.b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341470+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983361+0000 0 0 2026-02-17T21:22:59.028 INFO:tasks.workunit.client.0.trial031.stdout:1.a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341090+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983286+0000 0 0 2026-02-17T21:22:59.028 INFO:tasks.workunit.client.0.trial031.stdout:1.9 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341294+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983315+0000 0 0 2026-02-17T21:22:59.028 INFO:tasks.workunit.client.0.trial031.stdout:1.8 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.340967+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983302+0000 0 0 2026-02-17T21:22:59.028 INFO:tasks.workunit.client.0.trial031.stdout:1.7 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344971+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983373+0000 0 0 2026-02-17T21:22:59.028 INFO:tasks.workunit.client.0.trial031.stdout:1.6 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343790+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983335+0000 0 0 2026-02-17T21:22:59.029 INFO:tasks.workunit.client.0.trial031.stdout:1.5 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343964+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983341+0000 0 0 2026-02-17T21:22:59.029 INFO:tasks.workunit.client.0.trial031.stdout:1.4 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343986+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983354+0000 0 0 2026-02-17T21:22:59.029 INFO:tasks.workunit.client.0.trial031.stdout:1.3 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343765+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983348+0000 0 0 2026-02-17T21:22:59.029 INFO:tasks.workunit.client.0.trial031.stdout:1.2 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343766+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983322+0000 0 0 2026-02-17T21:22:59.029 INFO:tasks.workunit.client.0.trial031.stdout:1.c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343947+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983366+0000 0 0 2026-02-17T21:22:59.029 INFO:tasks.workunit.client.0.trial031.stdout:1.1 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341729+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983295+0000 0 0 2026-02-17T21:22:59.029 INFO:tasks.workunit.client.0.trial031.stdout:1.1c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341106+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983465+0000 0 0 2026-02-17T21:22:59.029 INFO:tasks.workunit.client.0.trial031.stdout:1.0 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341066+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983309+0000 0 0 2026-02-17T21:22:59.029 INFO:tasks.workunit.client.0.trial031.stdout:1.d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341537+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983329+0000 0 0 2026-02-17T21:22:59.029 INFO:tasks.workunit.client.0.trial031.stdout:1.e 3 0 3 0 0 2467 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:22:06.343911+0000 45'101 47:146 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983446+0000 0 0 2026-02-17T21:22:59.029 INFO:tasks.workunit.client.0.trial031.stdout:1.f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344345+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983413+0000 0 0 2026-02-17T21:22:59.029 INFO:tasks.workunit.client.0.trial031.stdout:1.10 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341474+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983439+0000 0 0 2026-02-17T21:22:59.030 INFO:tasks.workunit.client.0.trial031.stdout:1.11 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342237+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983379+0000 0 0 2026-02-17T21:22:59.030 INFO:tasks.workunit.client.0.trial031.stdout:1.12 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341306+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983400+0000 0 0 2026-02-17T21:22:59.030 INFO:tasks.workunit.client.0.trial031.stdout:1.13 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341120+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983471+0000 0 0 2026-02-17T21:22:59.030 INFO:tasks.workunit.client.0.trial031.stdout:1.14 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341229+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983477+0000 0 0 2026-02-17T21:22:59.030 INFO:tasks.workunit.client.0.trial031.stdout:1.15 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341182+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983406+0000 0 0 2026-02-17T21:22:59.030 INFO:tasks.workunit.client.0.trial031.stdout:1.16 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341319+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983490+0000 0 0 2026-02-17T21:22:59.030 INFO:tasks.workunit.client.0.trial031.stdout:1.17 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343856+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983458+0000 0 0 2026-02-17T21:22:59.030 INFO:tasks.workunit.client.0.trial031.stdout:1.18 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342096+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983393+0000 0 0 2026-02-17T21:22:59.030 INFO:tasks.workunit.client.0.trial031.stdout:1.19 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343794+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983433+0000 0 0 2026-02-17T21:22:59.030 INFO:tasks.workunit.client.0.trial031.stdout:1.1a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342036+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983386+0000 0 0 2026-02-17T21:22:59.030 INFO:tasks.workunit.client.0.trial031.stdout:1.1b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341167+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983426+0000 0 0 2026-02-17T21:22:59.030 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:22:59.031 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:22:59.031 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:22:59.035 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:794: TEST_divergent_3: kill_daemons td/divergent-priors KILL osd.1 2026-02-17T21:22:59.036 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:22:59.036 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:22:59.036 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:22:59.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:22:59.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:22:59.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:22:59.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:797: TEST_divergent_3: expfile=td/divergent-priors/exp.16076.out 2026-02-17T21:22:59.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:798: TEST_divergent_3: _objectstore_tool_nodown td/divergent-priors 1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.16076.out 2026-02-17T21:22:59.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-17T21:22:59.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T21:22:59.146 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T21:22:59.146 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T21:22:59.146 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-17T21:22:59.146 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.16076.out 2026-02-17T21:22:59.801 INFO:tasks.workunit.client.0.trial031.stderr:Exporting 1.0 info 1.0( v 28'100 (28'90,28'100] local-lis/les=50/51 n=3 ec=21/21 lis/c=50/50 les/c/f=51/51/0 sis=50) 2026-02-17T21:22:59.802 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:023dd3d2:::existing_12:head# 2026-02-17T21:22:59.802 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:04c40a0c:::existing_93:head# 2026-02-17T21:22:59.803 INFO:tasks.workunit.client.0.trial031.stderr:Read #1:064ff5fb:::existing_36:head# 2026-02-17T21:22:59.803 INFO:tasks.workunit.client.0.trial031.stderr:Export successful 2026-02-17T21:22:59.803 INFO:tasks.workunit.client.0.trial031.stdout: marking collection for removal 2026-02-17T21:22:59.803 INFO:tasks.workunit.client.0.trial031.stdout:setting '_remove' omap key 2026-02-17T21:22:59.803 INFO:tasks.workunit.client.0.trial031.stdout:finish_remove_pgs 1.0_head removing 1.0 2026-02-17T21:22:59.804 INFO:tasks.workunit.client.0.trial031.stderr:Remove successful 2026-02-17T21:23:00.341 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:799: TEST_divergent_3: _objectstore_tool_nodown td/divergent-priors 1 --op import --file td/divergent-priors/exp.16076.out 2026-02-17T21:23:00.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-17T21:23:00.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T21:23:00.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T21:23:00.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T21:23:00.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-17T21:23:00.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op import --file td/divergent-priors/exp.16076.out 2026-02-17T21:23:00.983 INFO:tasks.workunit.client.0.trial031.stdout:get_pg_num_history pg_num_history pg_num_history(e52 pg_nums {1={21=1,38=32}} deleted_pools ) 2026-02-17T21:23:00.983 INFO:tasks.workunit.client.0.trial031.stdout:Importing pgid 1.0 2026-02-17T21:23:00.983 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:023dd3d2:::existing_12:head# 2026-02-17T21:23:00.983 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:23:00.984 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:04c40a0c:::existing_93:head# 2026-02-17T21:23:00.984 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:23:00.984 INFO:tasks.workunit.client.0.trial031.stdout:Write #1:064ff5fb:::existing_36:head# 2026-02-17T21:23:00.985 INFO:tasks.workunit.client.0.trial031.stdout:snapset 0=[]:{} 2026-02-17T21:23:00.985 INFO:tasks.workunit.client.0.trial031.stdout:write_pg epoch 51 info 1.0( v 28'100 (28'90,28'100] local-lis/les=50/51 n=3 ec=21/21 lis/c=50/50 les/c/f=51/51/0 sis=50) 2026-02-17T21:23:00.985 INFO:tasks.workunit.client.0.trial031.stdout:Import successful 2026-02-17T21:23:01.521 INFO:tasks.workunit.client.0.trial031.stdout:reviving divergent 1 2026-02-17T21:23:01.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:801: TEST_divergent_3: echo 'reviving divergent 1' 2026-02-17T21:23:01.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:802: TEST_divergent_3: ceph pg dump pgs 2026-02-17T21:23:01.716 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:23:01.716 INFO:tasks.workunit.client.0.trial031.stdout:1.1f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343954+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983452+0000 0 0 2026-02-17T21:23:01.716 INFO:tasks.workunit.client.0.trial031.stdout:1.1e 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341401+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983484+0000 0 0 2026-02-17T21:23:01.716 INFO:tasks.workunit.client.0.trial031.stdout:1.1d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341205+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983419+0000 0 0 2026-02-17T21:23:01.716 INFO:tasks.workunit.client.0.trial031.stdout:1.b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341470+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983361+0000 0 0 2026-02-17T21:23:01.716 INFO:tasks.workunit.client.0.trial031.stdout:1.a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341090+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983286+0000 0 0 2026-02-17T21:23:01.716 INFO:tasks.workunit.client.0.trial031.stdout:1.9 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341294+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983315+0000 0 0 2026-02-17T21:23:01.717 INFO:tasks.workunit.client.0.trial031.stdout:1.8 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.340967+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983302+0000 0 0 2026-02-17T21:23:01.717 INFO:tasks.workunit.client.0.trial031.stdout:1.7 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344971+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983373+0000 0 0 2026-02-17T21:23:01.717 INFO:tasks.workunit.client.0.trial031.stdout:1.6 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343790+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983335+0000 0 0 2026-02-17T21:23:01.717 INFO:tasks.workunit.client.0.trial031.stdout:1.5 3 0 3 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343964+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983341+0000 0 0 2026-02-17T21:23:01.717 INFO:tasks.workunit.client.0.trial031.stdout:1.4 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343986+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983354+0000 0 0 2026-02-17T21:23:01.717 INFO:tasks.workunit.client.0.trial031.stdout:1.3 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343765+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983348+0000 0 0 2026-02-17T21:23:01.717 INFO:tasks.workunit.client.0.trial031.stdout:1.2 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343766+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983322+0000 0 0 2026-02-17T21:23:01.717 INFO:tasks.workunit.client.0.trial031.stdout:1.c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343947+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983366+0000 0 0 2026-02-17T21:23:01.717 INFO:tasks.workunit.client.0.trial031.stdout:1.1 4 0 4 0 0 2207 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341729+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983295+0000 0 0 2026-02-17T21:23:01.717 INFO:tasks.workunit.client.0.trial031.stdout:1.1c 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341106+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983465+0000 0 0 2026-02-17T21:23:01.717 INFO:tasks.workunit.client.0.trial031.stdout:1.0 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341066+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983309+0000 0 0 2026-02-17T21:23:01.717 INFO:tasks.workunit.client.0.trial031.stdout:1.d 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341537+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983329+0000 0 0 2026-02-17T21:23:01.718 INFO:tasks.workunit.client.0.trial031.stdout:1.e 3 0 3 0 0 2467 0 0 17 84 17 active+undersized+degraded 2026-02-17T21:22:06.343911+0000 45'101 47:146 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983446+0000 0 0 2026-02-17T21:23:01.718 INFO:tasks.workunit.client.0.trial031.stdout:1.f 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.344345+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983413+0000 0 0 2026-02-17T21:23:01.718 INFO:tasks.workunit.client.0.trial031.stdout:1.10 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341474+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983439+0000 0 0 2026-02-17T21:23:01.718 INFO:tasks.workunit.client.0.trial031.stdout:1.11 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342237+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983379+0000 0 0 2026-02-17T21:23:01.718 INFO:tasks.workunit.client.0.trial031.stdout:1.12 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341306+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983400+0000 0 0 2026-02-17T21:23:01.718 INFO:tasks.workunit.client.0.trial031.stdout:1.13 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341120+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983471+0000 0 0 2026-02-17T21:23:01.718 INFO:tasks.workunit.client.0.trial031.stdout:1.14 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341229+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983477+0000 0 0 2026-02-17T21:23:01.718 INFO:tasks.workunit.client.0.trial031.stdout:1.15 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341182+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983406+0000 0 0 2026-02-17T21:23:01.718 INFO:tasks.workunit.client.0.trial031.stdout:1.16 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341319+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983490+0000 0 0 2026-02-17T21:23:01.718 INFO:tasks.workunit.client.0.trial031.stdout:1.17 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343856+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983458+0000 0 0 2026-02-17T21:23:01.718 INFO:tasks.workunit.client.0.trial031.stdout:1.18 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342096+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983393+0000 0 0 2026-02-17T21:23:01.718 INFO:tasks.workunit.client.0.trial031.stdout:1.19 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.343794+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983433+0000 0 0 2026-02-17T21:23:01.719 INFO:tasks.workunit.client.0.trial031.stdout:1.1a 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.342036+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983386+0000 0 0 2026-02-17T21:23:01.719 INFO:tasks.workunit.client.0.trial031.stdout:1.1b 3 0 3 0 0 2206 0 0 16 84 16 active+undersized+degraded 2026-02-17T21:22:06.341167+0000 28'100 47:145 [0,2] 0 [0,2] 0 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-17T21:22:55.983426+0000 0 0 2026-02-17T21:23:01.719 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:23:01.719 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:23:01.719 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:23:01.727 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:803: TEST_divergent_3: activate_osd td/divergent-priors 1 2026-02-17T21:23:01.727 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:23:01.727 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:23:01.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T21:23:01.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:23:01.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-17T21:23:01.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:23:01.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:23:01.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:23:01.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:23:01.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-17T21:23:01.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-17T21:23:01.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:23:01.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:23:01.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:23:01.729 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:23:01.729 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:23:01.729 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:23:01.729 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:23:01.729 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:23:01.729 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:23:01.729 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:23:01.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:23:01.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:23:01.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:23:01.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:23:01.730 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:23:01.730 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:23:01.730 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:23:01.730 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:23:01.730 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:23:01.730 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:23:01.730 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-17T21:23:01.730 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:23:01.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T21:23:01.732 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:23:01.732 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-17T21:23:01.732 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T21:23:01.732 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:23:01.732 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:23:01.732 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:23:01.754 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:01.748+0000 7fda960f28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:01.756 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:01.748+0000 7fda960f28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:01.756 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:01.752+0000 7fda960f28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:01.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T21:23:01.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:23:01.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:23:01.927 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:23:01.928 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:23:01.928 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:23:01.928 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:01.928 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:23:01.928 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:01.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:23:02.131 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 50 up_thru 50 down_at 30 last_clean_interval [9,28) [v2:127.0.0.1:6818/3457322548,v1:127.0.0.1:6819/3457322548] [v2:127.0.0.1:6820/3457322548,v1:127.0.0.1:6821/3457322548] exists,up 2bdda522-e523-463c-b285-abb08be7107e 2026-02-17T21:23:02.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:23:02.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:23:02.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:23:02.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:804: TEST_divergent_3: wait_for_osd up 1 2026-02-17T21:23:02.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:23:02.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:23:02.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:23:02.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:23:02.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:02.132 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:23:02.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:23:02.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:02.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:23:02.336 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 50 up_thru 50 down_at 30 last_clean_interval [9,28) [v2:127.0.0.1:6818/3457322548,v1:127.0.0.1:6819/3457322548] [v2:127.0.0.1:6820/3457322548,v1:127.0.0.1:6821/3457322548] exists,up 2bdda522-e523-463c-b285-abb08be7107e 2026-02-17T21:23:02.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:23:02.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:23:02.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:23:02.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:806: TEST_divergent_3: sleep 20 2026-02-17T21:23:03.038 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:03.032+0000 7fda960f28c0 -1 Falling back to public interface 2026-02-17T21:23:06.005 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:06.000+0000 7fda960f28c0 -1 osd.1 52 log_to_monitors true 2026-02-17T21:23:22.338 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:807: TEST_divergent_3: get_asok_path osd.1 2026-02-17T21:23:22.338 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T21:23:22.338 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T21:23:22.339 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:23:22.339 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:23:22.340 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:23:22.340 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.1.asok 2026-02-17T21:23:22.340 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:807: TEST_divergent_3: CEPH_ARGS= 2026-02-17T21:23:22.340 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:807: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.1.asok dump_ops_in_flight 2026-02-17T21:23:22.394 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:23:22.394 INFO:tasks.workunit.client.0.trial031.stdout: "ops": [], 2026-02-17T21:23:22.394 INFO:tasks.workunit.client.0.trial031.stdout: "num_ops": 0 2026-02-17T21:23:22.394 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:23:22.400 INFO:tasks.workunit.client.0.trial031.stdout:allowing recovery 2026-02-17T21:23:22.400 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:809: TEST_divergent_3: echo 'allowing recovery' 2026-02-17T21:23:22.400 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:810: TEST_divergent_3: ceph pg dump pgs 2026-02-17T21:23:22.604 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:23:22.605 INFO:tasks.workunit.client.0.trial031.stdout:1.1f 3 0 0 0 0 2206 0 0 16 84 16 active+clean 2026-02-17T21:23:12.933465+0000 28'100 60:139 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:22.605 INFO:tasks.workunit.client.0.trial031.stdout:1.1e 3 0 0 0 0 2206 0 0 16 84 16 active+clean 2026-02-17T21:23:14.932319+0000 28'100 63:168 [0,1,2] 0 [0,1,2] 0 28'100 2026-02-17T21:23:13.933943+0000 28'100 2026-02-17T21:23:13.933943+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:22.605 INFO:tasks.workunit.client.0.trial031.stdout:1.1d 4 0 0 0 0 2824 0 0 16 84 16 active+clean 2026-02-17T21:23:17.153522+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:17.153448+0000 28'100 2026-02-17T21:23:17.153448+0000 0 0 periodic scrub scheduled @ 2026-02-19T00:04:56.190304+0000 4 0 2026-02-17T21:23:22.605 INFO:tasks.workunit.client.0.trial031.stdout:1.b 3 0 0 0 0 2206 0 0 16 84 16 active+clean 2026-02-17T21:23:12.932827+0000 28'100 60:139 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:22.605 INFO:tasks.workunit.client.0.trial031.stdout:1.a 3 0 0 0 0 2118 0 0 16 84 16 active+clean 2026-02-17T21:23:18.972149+0000 28'100 63:176 [0,1,2] 0 [0,1,2] 0 28'100 2026-02-17T21:23:18.972095+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T09:12:30.998120+0000 3 0 2026-02-17T21:23:22.605 INFO:tasks.workunit.client.0.trial031.stdout:1.9 5 0 0 0 0 3530 0 0 16 84 16 active+clean 2026-02-17T21:23:14.105596+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:14.105541+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T01:16:38.810676+0000 5 0 2026-02-17T21:23:22.605 INFO:tasks.workunit.client.0.trial031.stdout:1.8 3 0 0 0 0 2118 0 0 10 90 10 active+clean 2026-02-17T21:23:10.933198+0000 28'100 60:172 [1,0,2] 1 [1,0,2] 1 28'100 2026-02-17T21:23:10.933135+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T04:32:50.491225+0000 3 0 2026-02-17T21:23:22.605 INFO:tasks.workunit.client.0.trial031.stdout:1.7 3 0 0 0 0 2207 0 0 16 84 16 active+clean 2026-02-17T21:23:12.933486+0000 28'100 60:139 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:22.605 INFO:tasks.workunit.client.0.trial031.stdout:1.6 4 0 0 0 0 2913 0 0 17 84 17 active+clean 2026-02-17T21:23:14.932104+0000 52'101 63:163 [0,1,2] 0 [0,1,2] 0 52'101 2026-02-17T21:23:08.951468+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:22.606 INFO:tasks.workunit.client.0.trial031.stdout:1.5 6 0 0 0 0 4236 0 0 16 84 16 active+clean 2026-02-17T21:23:20.133209+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:20.133145+0000 0'0 2026-02-17T21:21:13.446509+0000 0 1 periodic scrub scheduled @ 2026-02-18T22:13:56.817399+0000 6 0 2026-02-17T21:23:22.606 INFO:tasks.workunit.client.0.trial031.stdout:1.4 4 0 0 0 0 2207 0 0 16 84 16 active+clean 2026-02-17T21:23:06.397184+0000 28'100 60:163 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:22.606 INFO:tasks.workunit.client.0.trial031.stdout:1.3 4 0 0 0 0 2207 0 0 16 84 16 active+clean 2026-02-17T21:23:12.933529+0000 28'100 60:139 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:22.606 INFO:tasks.workunit.client.0.trial031.stdout:1.2 1 0 0 0 0 706 0 0 16 84 16 active+clean 2026-02-17T21:23:20.011018+0000 28'100 63:176 [0,1,2] 0 [0,1,2] 0 28'100 2026-02-17T21:23:20.010963+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T01:54:04.316081+0000 1 0 2026-02-17T21:23:22.606 INFO:tasks.workunit.client.0.trial031.stdout:1.c 3 0 0 0 0 2118 0 0 14 86 14 active+clean 2026-02-17T21:23:07.992890+0000 28'100 60:172 [1,0,2] 1 [1,0,2] 1 28'100 2026-02-17T21:23:07.992810+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-18T23:14:08.075478+0000 3 0 2026-02-17T21:23:22.606 INFO:tasks.workunit.client.0.trial031.stdout:1.1 2 0 0 0 0 1412 0 0 16 84 16 active+clean 2026-02-17T21:23:13.066037+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:13.065978+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T00:58:42.277951+0000 2 0 2026-02-17T21:23:22.606 INFO:tasks.workunit.client.0.trial031.stdout:1.1c 2 0 0 0 0 1412 0 0 16 84 16 active+clean 2026-02-17T21:23:12.911304+0000 28'100 60:171 [1,0,2] 1 [1,0,2] 1 28'100 2026-02-17T21:23:12.911242+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T02:52:49.870989+0000 2 0 2026-02-17T21:23:22.606 INFO:tasks.workunit.client.0.trial031.stdout:1.0 3 0 0 0 0 2118 0 0 10 90 10 active+clean 2026-02-17T21:23:11.914835+0000 28'100 60:172 [1,0,2] 1 [1,0,2] 1 28'100 2026-02-17T21:23:11.914779+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-18T22:21:28.471353+0000 3 0 2026-02-17T21:23:22.606 INFO:tasks.workunit.client.0.trial031.stdout:1.d 3 0 0 0 0 2118 0 0 16 84 16 active+clean 2026-02-17T21:23:19.118296+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:19.118234+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T00:32:45.263348+0000 3 0 2026-02-17T21:23:22.606 INFO:tasks.workunit.client.0.trial031.stdout:1.e 4 0 0 0 0 3085 0 0 17 84 17 active+clean 2026-02-17T21:23:14.991151+0000 45'101 63:177 [0,1,2] 0 [0,1,2] 0 45'101 2026-02-17T21:23:14.991090+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T04:06:10.132817+0000 4 0 2026-02-17T21:23:22.606 INFO:tasks.workunit.client.0.trial031.stdout:1.f 3 0 0 0 0 2206 0 0 16 84 16 active+clean 2026-02-17T21:23:12.932946+0000 28'100 60:139 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:22.607 INFO:tasks.workunit.client.0.trial031.stdout:1.10 3 0 0 0 0 2118 0 0 10 90 10 active+clean 2026-02-17T21:23:14.908399+0000 28'100 60:172 [1,0,2] 1 [1,0,2] 1 28'100 2026-02-17T21:23:14.908336+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-18T22:17:37.333410+0000 3 0 2026-02-17T21:23:22.607 INFO:tasks.workunit.client.0.trial031.stdout:1.11 3 0 0 0 0 2118 0 0 16 84 16 active+clean 2026-02-17T21:23:15.125352+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:15.125289+0000 0'0 2026-02-17T21:21:13.446509+0000 0 1 periodic scrub scheduled @ 2026-02-19T07:46:10.119907+0000 3 0 2026-02-17T21:23:22.607 INFO:tasks.workunit.client.0.trial031.stdout:1.12 1 0 0 0 0 706 0 0 16 84 16 active+clean 2026-02-17T21:23:18.006141+0000 28'100 63:176 [0,1,2] 0 [0,1,2] 0 28'100 2026-02-17T21:23:18.006084+0000 28'100 2026-02-17T21:23:09.964414+0000 0 0 periodic scrub scheduled @ 2026-02-18T22:19:54.609026+0000 1 0 2026-02-17T21:23:22.607 INFO:tasks.workunit.client.0.trial031.stdout:1.13 3 0 0 0 0 2206 0 0 16 84 16 active+clean 2026-02-17T21:23:12.932956+0000 28'100 60:139 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:07.009884+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:22.607 INFO:tasks.workunit.client.0.trial031.stdout:1.14 3 0 0 0 0 2206 0 0 10 90 10 active+clean 2026-02-17T21:23:06.396835+0000 28'100 60:164 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:22.607 INFO:tasks.workunit.client.0.trial031.stdout:1.15 3 0 0 0 0 2118 0 0 16 84 16 active+clean 2026-02-17T21:23:18.107629+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:18.107570+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T06:56:39.556858+0000 3 0 2026-02-17T21:23:22.607 INFO:tasks.workunit.client.0.trial031.stdout:1.16 4 0 0 0 0 2824 0 0 16 84 16 active+clean 2026-02-17T21:23:15.950773+0000 28'100 63:176 [0,1,2] 0 [0,1,2] 0 28'100 2026-02-17T21:23:15.950690+0000 28'100 2026-02-17T21:23:15.950690+0000 0 0 periodic scrub scheduled @ 2026-02-19T03:54:37.658740+0000 4 0 2026-02-17T21:23:22.607 INFO:tasks.workunit.client.0.trial031.stdout:1.17 3 0 0 0 0 2206 0 0 16 84 16 active+clean 2026-02-17T21:23:12.933089+0000 28'100 60:139 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:22.607 INFO:tasks.workunit.client.0.trial031.stdout:1.18 3 0 0 0 0 2118 0 0 10 90 10 active+clean 2026-02-17T21:23:15.910141+0000 28'100 60:172 [1,0,2] 1 [1,0,2] 1 28'100 2026-02-17T21:23:15.910074+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T02:25:16.428291+0000 3 0 2026-02-17T21:23:22.607 INFO:tasks.workunit.client.0.trial031.stdout:1.19 2 0 0 0 0 1412 0 0 16 84 16 active+clean 2026-02-17T21:23:16.173499+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:16.173445+0000 0'0 2026-02-17T21:21:13.446509+0000 0 1 periodic scrub scheduled @ 2026-02-19T03:14:09.450831+0000 2 0 2026-02-17T21:23:22.607 INFO:tasks.workunit.client.0.trial031.stdout:1.1a 4 0 0 0 0 2824 0 0 16 84 16 active+clean 2026-02-17T21:23:16.956602+0000 28'100 63:176 [0,1,2] 0 [0,1,2] 0 28'100 2026-02-17T21:23:16.956539+0000 28'100 2026-02-17T21:23:16.956539+0000 0 0 periodic scrub scheduled @ 2026-02-18T23:44:44.237363+0000 4 0 2026-02-17T21:23:22.607 INFO:tasks.workunit.client.0.trial031.stdout:1.1b 3 0 0 0 0 2206 0 0 16 84 16 active+clean 2026-02-17T21:23:12.933277+0000 28'100 60:139 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:22.607 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:23:22.608 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:23:22.608 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:23:22.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:812: TEST_divergent_3: for i in $osds 2026-02-17T21:23:22.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:814: TEST_divergent_3: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-17T21:23:22.698 INFO:tasks.workunit.client.0.trial031.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-17T21:23:22.705 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:812: TEST_divergent_3: for i in $osds 2026-02-17T21:23:22.705 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:814: TEST_divergent_3: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-17T21:23:22.782 INFO:tasks.workunit.client.0.trial031.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-17T21:23:22.789 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:812: TEST_divergent_3: for i in $osds 2026-02-17T21:23:22.789 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:814: TEST_divergent_3: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-17T21:23:22.871 INFO:tasks.workunit.client.0.trial031.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-17T21:23:22.878 INFO:tasks.workunit.client.0.trial031.stdout:reading divergent objects 2026-02-17T21:23:22.879 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:817: TEST_divergent_3: echo 'reading divergent objects' 2026-02-17T21:23:22.879 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:818: TEST_divergent_3: ceph pg dump pgs 2026-02-17T21:23:23.075 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:23:23.075 INFO:tasks.workunit.client.0.trial031.stdout:1.1f 3 0 0 0 0 2206 0 0 16 84 16 active+clean 2026-02-17T21:23:16.932321+0000 28'100 64:169 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:23.075 INFO:tasks.workunit.client.0.trial031.stdout:1.1e 3 0 0 0 0 2206 0 0 16 84 16 active+clean 2026-02-17T21:23:14.932319+0000 28'100 63:168 [0,1,2] 0 [0,1,2] 0 28'100 2026-02-17T21:23:13.933943+0000 28'100 2026-02-17T21:23:13.933943+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:23.075 INFO:tasks.workunit.client.0.trial031.stdout:1.1d 4 0 0 0 0 2824 0 0 16 84 16 active+clean 2026-02-17T21:23:17.153522+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:17.153448+0000 28'100 2026-02-17T21:23:17.153448+0000 0 0 periodic scrub scheduled @ 2026-02-19T00:04:56.190304+0000 4 0 2026-02-17T21:23:23.075 INFO:tasks.workunit.client.0.trial031.stdout:1.b 3 0 0 0 0 2118 0 0 10 90 10 active+clean 2026-02-17T21:23:18.907163+0000 28'100 64:179 [1,2,0] 1 [1,2,0] 1 28'100 2026-02-17T21:23:18.907099+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T07:05:20.313555+0000 3 0 2026-02-17T21:23:23.075 INFO:tasks.workunit.client.0.trial031.stdout:1.a 3 0 0 0 0 2118 0 0 16 84 16 active+clean 2026-02-17T21:23:18.972149+0000 28'100 63:176 [0,1,2] 0 [0,1,2] 0 28'100 2026-02-17T21:23:18.972095+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T09:12:30.998120+0000 3 0 2026-02-17T21:23:23.075 INFO:tasks.workunit.client.0.trial031.stdout:1.9 5 0 0 0 0 3530 0 0 16 84 16 active+clean 2026-02-17T21:23:14.105596+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:14.105541+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T01:16:38.810676+0000 5 0 2026-02-17T21:23:23.076 INFO:tasks.workunit.client.0.trial031.stdout:1.8 3 0 0 0 0 2118 0 0 10 90 10 active+clean 2026-02-17T21:23:10.933198+0000 28'100 64:183 [1,0,2] 1 [1,0,2] 1 28'100 2026-02-17T21:23:10.933135+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T02:28:23.763184+0000 3 0 2026-02-17T21:23:23.076 INFO:tasks.workunit.client.0.trial031.stdout:1.7 3 0 0 0 0 2207 0 0 16 84 16 active+clean 2026-02-17T21:23:16.932187+0000 28'100 64:169 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:23.076 INFO:tasks.workunit.client.0.trial031.stdout:1.6 4 0 0 0 0 2913 0 0 17 84 17 active+clean 2026-02-17T21:23:14.932104+0000 52'101 63:163 [0,1,2] 0 [0,1,2] 0 52'101 2026-02-17T21:23:08.951468+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:23.076 INFO:tasks.workunit.client.0.trial031.stdout:1.5 6 0 0 0 0 4236 0 0 16 84 16 active+clean 2026-02-17T21:23:20.133209+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:20.133145+0000 0'0 2026-02-17T21:21:13.446509+0000 0 1 periodic scrub scheduled @ 2026-02-18T22:13:56.817399+0000 6 0 2026-02-17T21:23:23.076 INFO:tasks.workunit.client.0.trial031.stdout:1.4 2 0 0 0 0 1412 0 0 16 84 16 active+clean 2026-02-17T21:23:17.858700+0000 28'100 64:182 [1,0,2] 1 [1,0,2] 1 28'100 2026-02-17T21:23:17.858638+0000 28'100 2026-02-17T21:23:17.858638+0000 0 0 periodic scrub scheduled @ 2026-02-19T06:34:40.490698+0000 2 0 2026-02-17T21:23:23.076 INFO:tasks.workunit.client.0.trial031.stdout:1.3 4 0 0 0 0 2207 0 0 10 90 10 active+clean 2026-02-17T21:23:16.931100+0000 28'100 64:171 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:23.076 INFO:tasks.workunit.client.0.trial031.stdout:1.2 1 0 0 0 0 706 0 0 16 84 16 active+clean 2026-02-17T21:23:20.011018+0000 28'100 63:176 [0,1,2] 0 [0,1,2] 0 28'100 2026-02-17T21:23:20.010963+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T01:54:04.316081+0000 1 0 2026-02-17T21:23:23.076 INFO:tasks.workunit.client.0.trial031.stdout:1.c 3 0 0 0 0 2118 0 0 14 86 14 active+clean 2026-02-17T21:23:07.992890+0000 28'100 64:183 [1,0,2] 1 [1,0,2] 1 28'100 2026-02-17T21:23:07.992810+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T05:37:49.127789+0000 3 0 2026-02-17T21:23:23.076 INFO:tasks.workunit.client.0.trial031.stdout:1.1 2 0 0 0 0 1412 0 0 16 84 16 active+clean 2026-02-17T21:23:13.066037+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:13.065978+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T00:58:42.277951+0000 2 0 2026-02-17T21:23:23.076 INFO:tasks.workunit.client.0.trial031.stdout:1.1c 2 0 0 0 0 1412 0 0 16 84 16 active+clean 2026-02-17T21:23:12.911304+0000 28'100 64:182 [1,0,2] 1 [1,0,2] 1 28'100 2026-02-17T21:23:12.911242+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-18T23:39:58.259799+0000 2 0 2026-02-17T21:23:23.076 INFO:tasks.workunit.client.0.trial031.stdout:1.0 3 0 0 0 0 2118 0 0 10 90 10 active+clean 2026-02-17T21:23:11.914835+0000 28'100 64:183 [1,0,2] 1 [1,0,2] 1 28'100 2026-02-17T21:23:11.914779+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T03:23:12.690886+0000 3 0 2026-02-17T21:23:23.076 INFO:tasks.workunit.client.0.trial031.stdout:1.d 3 0 0 0 0 2118 0 0 16 84 16 active+clean 2026-02-17T21:23:19.118296+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:19.118234+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T00:32:45.263348+0000 3 0 2026-02-17T21:23:23.077 INFO:tasks.workunit.client.0.trial031.stdout:1.e 4 0 0 0 0 3085 0 0 17 84 17 active+clean 2026-02-17T21:23:14.991151+0000 45'101 63:177 [0,1,2] 0 [0,1,2] 0 45'101 2026-02-17T21:23:14.991090+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T04:06:10.132817+0000 4 0 2026-02-17T21:23:23.077 INFO:tasks.workunit.client.0.trial031.stdout:1.f 3 0 0 0 0 2206 0 0 16 84 16 active+clean 2026-02-17T21:23:16.931946+0000 28'100 64:169 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:23.077 INFO:tasks.workunit.client.0.trial031.stdout:1.10 3 0 0 0 0 2118 0 0 10 90 10 active+clean 2026-02-17T21:23:14.908399+0000 28'100 64:183 [1,0,2] 1 [1,0,2] 1 28'100 2026-02-17T21:23:14.908336+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T07:45:33.367807+0000 3 0 2026-02-17T21:23:23.077 INFO:tasks.workunit.client.0.trial031.stdout:1.11 3 0 0 0 0 2118 0 0 16 84 16 active+clean 2026-02-17T21:23:15.125352+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:15.125289+0000 0'0 2026-02-17T21:21:13.446509+0000 0 1 periodic scrub scheduled @ 2026-02-19T07:46:10.119907+0000 3 0 2026-02-17T21:23:23.077 INFO:tasks.workunit.client.0.trial031.stdout:1.12 1 0 0 0 0 706 0 0 16 84 16 active+clean 2026-02-17T21:23:18.006141+0000 28'100 63:176 [0,1,2] 0 [0,1,2] 0 28'100 2026-02-17T21:23:18.006084+0000 28'100 2026-02-17T21:23:09.964414+0000 0 0 periodic scrub scheduled @ 2026-02-18T22:19:54.609026+0000 1 0 2026-02-17T21:23:23.077 INFO:tasks.workunit.client.0.trial031.stdout:1.13 3 0 0 0 0 2118 0 0 10 90 10 active+clean 2026-02-17T21:23:19.913295+0000 28'100 64:187 [1,2,0] 1 [1,2,0] 1 28'100 2026-02-17T21:23:19.913215+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T02:42:40.106502+0000 3 0 2026-02-17T21:23:23.077 INFO:tasks.workunit.client.0.trial031.stdout:1.14 3 0 0 0 0 2118 0 0 10 90 10 active+clean 2026-02-17T21:23:16.874873+0000 28'100 64:183 [1,0,2] 1 [1,0,2] 1 28'100 2026-02-17T21:23:16.874810+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T03:43:19.679019+0000 3 0 2026-02-17T21:23:23.077 INFO:tasks.workunit.client.0.trial031.stdout:1.15 3 0 0 0 0 2118 0 0 16 84 16 active+clean 2026-02-17T21:23:18.107629+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:18.107570+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T06:56:39.556858+0000 3 0 2026-02-17T21:23:23.077 INFO:tasks.workunit.client.0.trial031.stdout:1.16 4 0 0 0 0 2824 0 0 16 84 16 active+clean 2026-02-17T21:23:15.950773+0000 28'100 63:176 [0,1,2] 0 [0,1,2] 0 28'100 2026-02-17T21:23:15.950690+0000 28'100 2026-02-17T21:23:15.950690+0000 0 0 periodic scrub scheduled @ 2026-02-19T03:54:37.658740+0000 4 0 2026-02-17T21:23:23.077 INFO:tasks.workunit.client.0.trial031.stdout:1.17 3 0 0 0 0 2206 0 0 16 84 16 active+clean 2026-02-17T21:23:16.932515+0000 28'100 64:169 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-17T21:21:13.446509+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 queued for scrub 0 0 2026-02-17T21:23:23.077 INFO:tasks.workunit.client.0.trial031.stdout:1.18 3 0 0 0 0 2118 0 0 10 90 10 active+clean 2026-02-17T21:23:15.910141+0000 28'100 64:183 [1,0,2] 1 [1,0,2] 1 28'100 2026-02-17T21:23:15.910074+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T07:30:29.258340+0000 3 0 2026-02-17T21:23:23.077 INFO:tasks.workunit.client.0.trial031.stdout:1.19 2 0 0 0 0 1412 0 0 16 84 16 active+clean 2026-02-17T21:23:16.173499+0000 28'100 63:155 [2,0,1] 2 [2,0,1] 2 28'100 2026-02-17T21:23:16.173445+0000 0'0 2026-02-17T21:21:13.446509+0000 0 1 periodic scrub scheduled @ 2026-02-19T03:14:09.450831+0000 2 0 2026-02-17T21:23:23.077 INFO:tasks.workunit.client.0.trial031.stdout:1.1a 4 0 0 0 0 2824 0 0 16 84 16 active+clean 2026-02-17T21:23:16.956602+0000 28'100 63:176 [0,1,2] 0 [0,1,2] 0 28'100 2026-02-17T21:23:16.956539+0000 28'100 2026-02-17T21:23:16.956539+0000 0 0 periodic scrub scheduled @ 2026-02-18T23:44:44.237363+0000 4 0 2026-02-17T21:23:23.078 INFO:tasks.workunit.client.0.trial031.stdout:1.1b 5 0 0 0 0 3530 0 0 10 90 10 active+clean 2026-02-17T21:23:20.899413+0000 28'100 64:181 [1,2,0] 1 [1,2,0] 1 28'100 2026-02-17T21:23:20.899336+0000 0'0 2026-02-17T21:21:13.446509+0000 0 0 periodic scrub scheduled @ 2026-02-19T01:00:54.759658+0000 5 0 2026-02-17T21:23:23.078 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:23:23.078 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:23:23.078 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:23:23.086 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: expr 2 + 2 2026-02-17T21:23:23.087 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: seq 1 4 2026-02-17T21:23:23.088 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:23:23.088 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:821: TEST_divergent_3: rados -p test get existing_1 td/divergent-priors/existing 2026-02-17T21:23:23.109 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:23:23.109 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:821: TEST_divergent_3: rados -p test get existing_2 td/divergent-priors/existing 2026-02-17T21:23:23.137 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:23:23.137 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:821: TEST_divergent_3: rados -p test get existing_3 td/divergent-priors/existing 2026-02-17T21:23:23.163 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:23:23.163 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:821: TEST_divergent_3: rados -p test get existing_4 td/divergent-priors/existing 2026-02-17T21:23:23.188 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: seq 1 2 2026-02-17T21:23:23.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-17T21:23:23.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:825: TEST_divergent_3: rados -p test get newobject_1 td/divergent-priors/existing 2026-02-17T21:23:23.208 INFO:tasks.workunit.client.0.trial031.stderr:error getting test/newobject_1: (2) No such file or directory 2026-02-17T21:23:23.210 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-17T21:23:23.210 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:825: TEST_divergent_3: rados -p test get newobject_2 td/divergent-priors/existing 2026-02-17T21:23:23.236 INFO:tasks.workunit.client.0.trial031.stderr:error getting test/newobject_2: (2) No such file or directory 2026-02-17T21:23:23.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:827: TEST_divergent_3: rm -f td/divergent-priors/existing 2026-02-17T21:23:23.240 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:829: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2026-02-17T21:23:23.241 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:829: TEST_divergent_3: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-17T21:23:23.245 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 20 _merge_object_divergent_entries: merging hoid 1:97db2a0f:::existing_2:head entries: 28'104 (28'2) modify 1:97db2a0f:::existing_2:head by client.4563.0:1 2026-02-17T21:21:27.565534+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:23:23.245 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 20 _merge_object_divergent_entries: keeping 28'104 (28'2) modify 1:97db2a0f:::existing_2:head by client.4563.0:1 2026-02-17T21:21:27.565534+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:23:23.245 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-17T21:23:23.245 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version: 28'2 first_divergent_update: 28'104 last_divergent_update: 28'104 2026-02-17T21:23:23.245 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head has no more recent entries in log 2026-02-17T21:23:23.246 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head must be rolled back or recovered, attempting to rollback 2026-02-17T21:23:23.246 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head olog_can_rollback_to: 28'106 2026-02-17T21:23:23.246 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot rollback 28'104 (28'2) modify 1:97db2a0f:::existing_2:head by client.4563.0:1 2026-02-17T21:21:27.565534+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:23:23.246 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot roll back, removing and adding to missing 2026-02-17T21:23:23.246 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version 28'2 <= info.log_tail 28'89 2026-02-17T21:23:23.246 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d4a9d5640 20 _merge_object_divergent_entries: merging hoid 1:70b3bf12:::existing_4:head entries: 28'106 (28'4) delete 1:70b3bf12:::existing_4:head by client.4569.0:1 2026-02-17T21:21:27.572484+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: false, new_object: false 2026-02-17T21:23:23.246 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d4a9d5640 20 _merge_object_divergent_entries: keeping 28'106 (28'4) delete 1:70b3bf12:::existing_4:head by client.4569.0:1 2026-02-17T21:21:27.572484+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: false, new_object: false 2026-02-17T21:23:23.246 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d4a9d5640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-17T21:23:23.246 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d4a9d5640 10 _merge_object_divergent_entries: hoid 1:70b3bf12:::existing_4:head prior_version: 28'4 first_divergent_update: 28'106 last_divergent_update: 28'106 2026-02-17T21:23:23.246 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d4a9d5640 10 _merge_object_divergent_entries: more recent entry found: 45'101 (28'4) modify 1:70b3bf12:::existing_4:head by client.4692.0:1 2026-02-17T21:22:20.875746+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false, already merged 2026-02-17T21:23:23.246 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d4a9d5640 20 _merge_object_divergent_entries: merging hoid 1:75ac5091:::newobject_2:head entries: 28'102 (0'0) modify 1:75ac5091:::newobject_2:head by client.4561.0:1 2026-02-17T21:21:27.564348+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-17T21:23:23.246 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d4a9d5640 20 _merge_object_divergent_entries: keeping 28'102 (0'0) modify 1:75ac5091:::newobject_2:head by client.4561.0:1 2026-02-17T21:21:27.564348+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-17T21:23:23.247 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d4a9d5640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-17T21:23:23.247 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d4a9d5640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version: 0'0 first_divergent_update: 28'102 last_divergent_update: 28'102 2026-02-17T21:23:23.247 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d4a9d5640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head has no more recent entries in log 2026-02-17T21:23:23.247 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d4a9d5640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version or op type indicates creation, deleting 2026-02-17T21:23:23.247 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 20 _merge_object_divergent_entries: merging hoid 1:8881e6d8:::existing_3:head entries: 28'105 (28'3) delete 1:8881e6d8:::existing_3:head by client.4568.0:1 2026-02-17T21:21:27.571927+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: false, new_object: false 2026-02-17T21:23:23.247 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 20 _merge_object_divergent_entries: keeping 28'105 (28'3) delete 1:8881e6d8:::existing_3:head by client.4568.0:1 2026-02-17T21:21:27.571927+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: false, new_object: false 2026-02-17T21:23:23.247 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid object_not_in_store: 1 2026-02-17T21:23:23.247 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version: 28'3 first_divergent_update: 28'105 last_divergent_update: 28'105 2026-02-17T21:23:23.247 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head has no more recent entries in log 2026-02-17T21:23:23.247 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head must be rolled back or recovered, attempting to rollback 2026-02-17T21:23:23.247 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head olog_can_rollback_to: 28'106 2026-02-17T21:23:23.247 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot rollback 28'105 (28'3) delete 1:8881e6d8:::existing_3:head by client.4568.0:1 2026-02-17T21:21:27.571927+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: false, new_object: false 2026-02-17T21:23:23.248 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot roll back, removing and adding to missing 2026-02-17T21:23:23.248 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d491d2640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version 28'3 <= info.log_tail 28'84 2026-02-17T21:23:23.248 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 20 _merge_object_divergent_entries: merging hoid 1:d858c035:::existing_1:head entries: 28'103 (28'1) modify 1:d858c035:::existing_1:head by client.4562.0:1 2026-02-17T21:21:27.564533+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:23:23.248 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 20 _merge_object_divergent_entries: keeping 28'103 (28'1) modify 1:d858c035:::existing_1:head by client.4562.0:1 2026-02-17T21:21:27.564533+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:23:23.256 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-17T21:23:23.256 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version: 28'1 first_divergent_update: 28'103 last_divergent_update: 28'103 2026-02-17T21:23:23.256 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head has no more recent entries in log 2026-02-17T21:23:23.256 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head must be rolled back or recovered, attempting to rollback 2026-02-17T21:23:23.256 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head olog_can_rollback_to: 28'106 2026-02-17T21:23:23.256 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot rollback 28'103 (28'1) modify 1:d858c035:::existing_1:head by client.4562.0:1 2026-02-17T21:21:27.564533+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:23:23.256 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot roll back, removing and adding to missing 2026-02-17T21:23:23.256 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version 28'1 <= info.log_tail 28'90 2026-02-17T21:23:23.256 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 20 _merge_object_divergent_entries: merging hoid 1:da59f6e7:::newobject_1:head entries: 28'101 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4559.0:1 2026-02-17T21:21:27.564025+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-17T21:23:23.256 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 20 _merge_object_divergent_entries: keeping 28'101 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4559.0:1 2026-02-17T21:21:27.564025+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-17T21:23:23.256 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-17T21:23:23.257 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version: 0'0 first_divergent_update: 28'101 last_divergent_update: 28'101 2026-02-17T21:23:23.257 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head has no more recent entries in log 2026-02-17T21:23:23.257 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:22:58.016+0000 7f9d481d0640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version or op type indicates creation, deleting 2026-02-17T21:23:23.257 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:831: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2026-02-17T21:23:23.257 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:831: TEST_divergent_3: grep -q '_merge_object_divergent_entries: more recent entry found:' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-17T21:23:23.261 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:837: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2026-02-17T21:23:23.262 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:837: TEST_divergent_3: grep -q '_merge_object_divergent_entries.*prior_version or op type indicates creation' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-17T21:23:23.265 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:842: TEST_divergent_3: echo success 2026-02-17T21:23:23.266 INFO:tasks.workunit.client.0.trial031.stdout:success 2026-02-17T21:23:23.267 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:844: TEST_divergent_3: rm td/divergent-priors/td/divergent-priors/exp.16076.out 2026-02-17T21:23:23.267 INFO:tasks.workunit.client.0.trial031.stderr:rm: cannot remove 'td/divergent-priors/td/divergent-priors/exp.16076.out': No such file or directory 2026-02-17T21:23:23.267 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:846: TEST_divergent_3: delete_pool test 2026-02-17T21:23:23.267 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=test 2026-02-17T21:23:23.267 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-17T21:23:23.509 INFO:tasks.workunit.client.0.trial031.stderr:pool 'test' does not exist 2026-02-17T21:23:23.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:847: TEST_divergent_3: kill_daemons td/divergent-priors 2026-02-17T21:23:23.522 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:23:23.522 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:23:23.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:23:23.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:23:23.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:23:31.839 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:23:31.839 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-17T21:23:31.840 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-17T21:23:31.840 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:23:31.840 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-17T21:23:31.840 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:23:31.841 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:23:31.841 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:23:31.841 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:23:31.841 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:23:31.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:23:31.848 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:23:31.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:23:31.849 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:23:31.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:23:31.850 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:23:31.850 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:23:31.851 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:23:31.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:23:31.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:23:31.852 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:23:31.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:23:31.854 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:23:31.855 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-17T21:23:31.883 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:23:31.883 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:23:31.883 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:23:31.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.16076 2026-02-17T21:23:31.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:23:31.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:23:31.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-17T21:23:31.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-17T21:23:31.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2026-02-17T21:23:31.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2026-02-17T21:23:31.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-17T21:23:31.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:23:31.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-17T21:23:31.886 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:23:31.886 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:23:31.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:23:31.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:23:31.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:23:31.887 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:23:31.888 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:23:31.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:23:31.889 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:23:31.890 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:23:31.890 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:23:31.890 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:23:31.891 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:23:31.891 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:23:31.892 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:23:31.892 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:23:31.893 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:23:31.894 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:23:31.894 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-17T21:23:31.895 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:23:31.895 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:23:31.895 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:23:31.896 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.16076 2026-02-17T21:23:31.896 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:23:31.896 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:23:31.896 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2026-02-17T21:23:31.898 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T21:23:31.898 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:23:31.898 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:23:31.898 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.16076 2026-02-17T21:23:31.900 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T21:23:31.901 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T21:23:31.901 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T21:23:31.901 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-17T21:23:31.901 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_ec td/divergent-priors 2026-02-17T21:23:31.901 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:228: TEST_divergent_ec: local dir=td/divergent-priors 2026-02-17T21:23:31.901 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:231: TEST_divergent_ec: local dummyfile=/etc/fstab 2026-02-17T21:23:31.901 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:232: TEST_divergent_ec: local dummyfile2=/etc/resolv.conf 2026-02-17T21:23:31.901 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:234: TEST_divergent_ec: local num_osds=3 2026-02-17T21:23:31.902 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:235: TEST_divergent_ec: expr 3 - 1 2026-02-17T21:23:31.902 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:235: TEST_divergent_ec: seq 0 2 2026-02-17T21:23:31.903 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:235: TEST_divergent_ec: local 'osds=0 2026-02-17T21:23:31.904 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:23:31.904 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:23:31.904 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:236: TEST_divergent_ec: run_mon td/divergent-priors a 2026-02-17T21:23:31.904 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2026-02-17T21:23:31.904 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T21:23:31.904 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T21:23:31.904 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T21:23:31.904 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2026-02-17T21:23:31.904 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-17T21:23:31.936 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T21:23:31.936 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:23:31.936 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:23:31.936 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:23:31.937 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:23:31.937 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:23:31.937 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:23:31.937 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T21:23:31.968 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T21:23:31.969 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T21:23:31.969 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:23:31.969 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:23:31.969 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T21:23:31.969 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T21:23:31.970 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:23:31.970 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:23:31.971 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:23:31.971 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:23:31.971 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:23:31.971 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:23:31.971 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-mon.a.asok 2026-02-17T21:23:31.971 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:23:31.971 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.16076/ceph-mon.a.asok config get fsid 2026-02-17T21:23:32.038 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T21:23:32.038 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:23:32.038 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:23:32.038 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T21:23:32.039 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T21:23:32.039 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:23:32.040 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:23:32.040 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:23:32.040 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:23:32.040 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:23:32.040 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:23:32.040 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-mon.a.asok 2026-02-17T21:23:32.040 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:23:32.040 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.16076/ceph-mon.a.asok config get mon_host 2026-02-17T21:23:32.095 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:237: TEST_divergent_ec: run_mgr td/divergent-priors x 2026-02-17T21:23:32.095 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/divergent-priors 2026-02-17T21:23:32.095 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T21:23:32.096 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T21:23:32.096 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T21:23:32.096 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/divergent-priors/x 2026-02-17T21:23:32.096 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T21:23:32.211 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T21:23:32.211 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:23:32.211 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:23:32.211 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:23:32.212 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:23:32.212 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:23:32.212 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:23:32.213 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:23:32.213 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:23:32.234 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:238: TEST_divergent_ec: for i in $osds 2026-02-17T21:23:32.234 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:240: TEST_divergent_ec: run_osd td/divergent-priors 0 2026-02-17T21:23:32.235 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-17T21:23:32.235 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:23:32.235 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T21:23:32.235 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:23:32.235 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/0 2026-02-17T21:23:32.235 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:23:32.235 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:23:32.235 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:23:32.235 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:23:32.235 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-17T21:23:32.235 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-17T21:23:32.236 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:23:32.236 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:23:32.236 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:23:32.236 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:23:32.236 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:23:32.236 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:23:32.236 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:23:32.236 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:23:32.236 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:23:32.236 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:23:32.236 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:23:32.236 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:23:32.236 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:23:32.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:23:32.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:23:32.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:23:32.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:23:32.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:23:32.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:23:32.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:23:32.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:23:32.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:23:32.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/0 2026-02-17T21:23:32.238 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:23:32.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=27052348-33c5-4b28-9883-30bfd1415320 2026-02-17T21:23:32.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 27052348-33c5-4b28-9883-30bfd1415320' 2026-02-17T21:23:32.239 INFO:tasks.workunit.client.0.trial031.stdout:add osd0 27052348-33c5-4b28-9883-30bfd1415320 2026-02-17T21:23:32.239 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:23:32.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBU3JRp+pjaDhAAPO5xzIoKc2vxgKxWyBRhFg== 2026-02-17T21:23:32.250 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBU3JRp+pjaDhAAPO5xzIoKc2vxgKxWyBRhFg=="}' 2026-02-17T21:23:32.250 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 27052348-33c5-4b28-9883-30bfd1415320 -i td/divergent-priors/0/new.json 2026-02-17T21:23:32.361 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:23:32.369 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/0/new.json 2026-02-17T21:23:32.370 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBU3JRp+pjaDhAAPO5xzIoKc2vxgKxWyBRhFg== --osd-uuid 27052348-33c5-4b28-9883-30bfd1415320 2026-02-17T21:23:32.388 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:32.388+0000 7f0dac7a58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:32.389 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:32.388+0000 7f0dac7a58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:32.390 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:32.388+0000 7f0dac7a58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:32.391 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:32.388+0000 7f0dac7a58c0 -1 bdev(0x5604170edc00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-17T21:23:32.391 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:32.388+0000 7f0dac7a58c0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-17T21:23:34.984 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-17T21:23:34.985 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:23:34.986 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T21:23:34.986 INFO:tasks.workunit.client.0.trial031.stdout:adding osd0 key to auth repository 2026-02-17T21:23:34.986 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:23:35.270 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T21:23:35.271 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:23:35.271 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:23:35.271 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:23:35.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:23:35.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:23:35.299 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:35.296+0000 7f7c69a1b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:35.301 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:35.300+0000 7f7c69a1b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:35.302 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:35.300+0000 7f7c69a1b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:35.470 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T21:23:35.471 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:23:35.471 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:23:35.471 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:23:35.471 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:23:35.471 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:35.471 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:23:35.471 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:23:35.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:35.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:23:35.678 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:23:36.679 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:23:36.680 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:36.680 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:23:36.680 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:23:36.681 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:36.681 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:23:36.781 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:36.779+0000 7f7c69a1b8c0 -1 Falling back to public interface 2026-02-17T21:23:36.882 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:23:37.883 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:23:37.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:37.884 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:23:37.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:23:37.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:37.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:23:38.091 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:23:39.092 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:23:39.092 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:39.092 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:23:39.093 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:23:39.093 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:39.093 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:23:39.294 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:23:40.004 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:40.003+0000 7f7c69a1b8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T21:23:40.296 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:23:40.296 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:40.296 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:23:40.296 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:23:40.297 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:40.297 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:23:40.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:23:41.503 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:23:41.504 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:41.504 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:23:41.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:23:41.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:41.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:23:41.715 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1852916604,v1:127.0.0.1:6803/1852916604] [v2:127.0.0.1:6804/1852916604,v1:127.0.0.1:6805/1852916604] exists,up 27052348-33c5-4b28-9883-30bfd1415320 2026-02-17T21:23:41.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:23:41.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:23:41.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:23:41.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:238: TEST_divergent_ec: for i in $osds 2026-02-17T21:23:41.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:240: TEST_divergent_ec: run_osd td/divergent-priors 1 2026-02-17T21:23:41.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-17T21:23:41.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:23:41.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T21:23:41.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:23:41.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/1 2026-02-17T21:23:41.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:23:41.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:23:41.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:23:41.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:23:41.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-17T21:23:41.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-17T21:23:41.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:23:41.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:23:41.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:23:41.720 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:23:41.720 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:23:41.720 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:23:41.720 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:23:41.721 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:23:41.721 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:23:41.721 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:23:41.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:23:41.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:23:41.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:23:41.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:23:41.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:23:41.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:23:41.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:23:41.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:23:41.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:23:41.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:23:41.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:23:41.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:23:41.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/1 2026-02-17T21:23:41.722 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:23:41.723 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=7650e991-0aa1-446e-810d-5162931dd850 2026-02-17T21:23:41.723 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 7650e991-0aa1-446e-810d-5162931dd850' 2026-02-17T21:23:41.723 INFO:tasks.workunit.client.0.trial031.stdout:add osd1 7650e991-0aa1-446e-810d-5162931dd850 2026-02-17T21:23:41.723 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:23:41.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBd3JRpSPukKxAAYIZHxdkwdiddYodL7CF/vg== 2026-02-17T21:23:41.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBd3JRpSPukKxAAYIZHxdkwdiddYodL7CF/vg=="}' 2026-02-17T21:23:41.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 7650e991-0aa1-446e-810d-5162931dd850 -i td/divergent-priors/1/new.json 2026-02-17T21:23:41.944 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:23:41.953 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/1/new.json 2026-02-17T21:23:41.954 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBd3JRpSPukKxAAYIZHxdkwdiddYodL7CF/vg== --osd-uuid 7650e991-0aa1-446e-810d-5162931dd850 2026-02-17T21:23:41.972 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:41.971+0000 7ff49e2538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:41.974 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:41.971+0000 7ff49e2538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:41.975 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:41.971+0000 7ff49e2538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:41.975 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:41.975+0000 7ff49e2538c0 -1 bdev(0x55b2e8d0fc00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-17T21:23:41.975 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:41.975+0000 7ff49e2538c0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-17T21:23:44.268 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-17T21:23:44.268 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:23:44.269 INFO:tasks.workunit.client.0.trial031.stdout:adding osd1 key to auth repository 2026-02-17T21:23:44.269 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T21:23:44.269 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:23:44.557 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:23:44.558 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T21:23:44.558 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:23:44.558 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:23:44.558 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:23:44.558 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:23:44.585 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:44.583+0000 7f826b6098c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:44.587 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:44.583+0000 7f826b6098c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:44.588 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:44.587+0000 7f826b6098c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:44.753 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T21:23:44.753 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:23:44.753 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:23:44.753 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:23:44.753 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:23:44.754 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:44.754 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:23:44.755 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:23:44.755 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:44.755 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:23:44.952 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:23:45.953 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:23:45.954 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:45.954 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:23:45.955 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:23:45.955 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:45.955 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:23:46.077 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:46.075+0000 7f826b6098c0 -1 Falling back to public interface 2026-02-17T21:23:46.158 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:23:47.160 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:23:47.160 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:47.161 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:23:47.161 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:23:47.161 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:47.161 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:23:47.360 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:23:48.362 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:23:48.362 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:48.362 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:23:48.363 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:23:48.363 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:48.363 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:23:48.569 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:23:49.075 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:49.071+0000 7f826b6098c0 -1 osd.1 0 log_to_monitors true 2026-02-17T21:23:49.571 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:23:49.571 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:49.571 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:23:49.572 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:23:49.572 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:49.572 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:23:49.767 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3438498595,v1:127.0.0.1:6811/3438498595] [v2:127.0.0.1:6812/3438498595,v1:127.0.0.1:6813/3438498595] exists,up 7650e991-0aa1-446e-810d-5162931dd850 2026-02-17T21:23:49.771 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:23:49.771 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:23:49.771 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:23:49.771 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:238: TEST_divergent_ec: for i in $osds 2026-02-17T21:23:49.771 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:240: TEST_divergent_ec: run_osd td/divergent-priors 2 2026-02-17T21:23:49.771 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-17T21:23:49.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:23:49.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T21:23:49.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:23:49.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/2 2026-02-17T21:23:49.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:23:49.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:23:49.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:23:49.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:23:49.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-17T21:23:49.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-17T21:23:49.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:23:49.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:23:49.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:23:49.773 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:23:49.773 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:23:49.773 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:23:49.773 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:23:49.773 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:23:49.773 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:23:49.773 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:23:49.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:23:49.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:23:49.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:23:49.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:23:49.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:23:49.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:23:49.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:23:49.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:23:49.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:23:49.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:23:49.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:23:49.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:23:49.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/2 2026-02-17T21:23:49.774 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:23:49.775 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a63bbb72-c70c-48fa-b386-d5e9b16de7b4 2026-02-17T21:23:49.775 INFO:tasks.workunit.client.0.trial031.stdout:add osd2 a63bbb72-c70c-48fa-b386-d5e9b16de7b4 2026-02-17T21:23:49.776 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 a63bbb72-c70c-48fa-b386-d5e9b16de7b4' 2026-02-17T21:23:49.776 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:23:49.786 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBl3JRpqvXaLhAAPvVN8a70wTrH2AgLrt2fVQ== 2026-02-17T21:23:49.787 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBl3JRpqvXaLhAAPvVN8a70wTrH2AgLrt2fVQ=="}' 2026-02-17T21:23:49.787 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a63bbb72-c70c-48fa-b386-d5e9b16de7b4 -i td/divergent-priors/2/new.json 2026-02-17T21:23:50.000 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:23:50.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/2/new.json 2026-02-17T21:23:50.010 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBl3JRpqvXaLhAAPvVN8a70wTrH2AgLrt2fVQ== --osd-uuid a63bbb72-c70c-48fa-b386-d5e9b16de7b4 2026-02-17T21:23:50.028 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:50.027+0000 7fc45921a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:50.030 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:50.027+0000 7fc45921a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:50.031 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:50.027+0000 7fc45921a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:50.031 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:50.027+0000 7fc45921a8c0 -1 bdev(0x558b80157c00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-17T21:23:50.031 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:50.031+0000 7fc45921a8c0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-17T21:23:52.288 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-17T21:23:52.288 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:23:52.289 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T21:23:52.289 INFO:tasks.workunit.client.0.trial031.stdout:adding osd2 key to auth repository 2026-02-17T21:23:52.289 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:23:52.566 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:23:52.567 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T21:23:52.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:23:52.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:23:52.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:23:52.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:23:52.594 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:52.591+0000 7fbb954cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:52.596 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:52.595+0000 7fbb954cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:52.597 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:52.595+0000 7fbb954cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:23:52.767 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T21:23:52.767 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:23:52.767 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:23:52.767 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:23:52.767 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:23:52.767 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:52.768 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:23:52.768 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:23:52.768 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:52.768 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:23:52.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:23:53.969 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:23:53.969 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:53.969 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:23:53.970 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:23:53.970 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:53.970 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:23:54.179 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:23:54.309 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:54.307+0000 7fbb954cc8c0 -1 Falling back to public interface 2026-02-17T21:23:55.180 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:23:55.181 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:55.181 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:23:55.182 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:23:55.182 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:55.182 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:23:55.383 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:23:56.384 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:23:56.384 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:56.385 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:23:56.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:23:56.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:56.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:23:56.587 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:23:57.291 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:23:57.287+0000 7fbb954cc8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T21:23:57.588 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:23:57.589 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:57.589 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:23:57.590 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:23:57.590 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:57.590 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:23:57.785 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:23:58.786 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:23:58.787 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:23:58.787 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:23:58.788 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:23:58.788 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:23:58.788 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:23:58.988 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/444752357,v1:127.0.0.1:6819/444752357] [v2:127.0.0.1:6820/444752357,v1:127.0.0.1:6821/444752357] exists,up a63bbb72-c70c-48fa-b386-d5e9b16de7b4 2026-02-17T21:23:58.989 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:23:58.989 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:23:58.990 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:23:58.990 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:243: TEST_divergent_ec: ceph osd set noout 2026-02-17T21:23:59.292 INFO:tasks.workunit.client.0.trial031.stderr:noout is set 2026-02-17T21:23:59.304 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:244: TEST_divergent_ec: ceph osd set noin 2026-02-17T21:23:59.597 INFO:tasks.workunit.client.0.trial031.stderr:noin is set 2026-02-17T21:23:59.609 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:245: TEST_divergent_ec: ceph osd set nodown 2026-02-17T21:23:59.901 INFO:tasks.workunit.client.0.trial031.stderr:nodown is set 2026-02-17T21:23:59.914 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:246: TEST_divergent_ec: create_ec_pool test true k=2 m=1 2026-02-17T21:23:59.914 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=test 2026-02-17T21:23:59.914 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T21:23:59.914 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=true 2026-02-17T21:23:59.914 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T21:23:59.914 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-17T21:24:00.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool test 1 1 erasure myprofile 2026-02-17T21:24:00.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 erasure myprofile 2026-02-17T21:24:00.559 INFO:tasks.workunit.client.0.trial031.stderr:pool 'test' already exists 2026-02-17T21:24:00.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T21:24:01.569 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' true = true ']' 2026-02-17T21:24:01.570 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2508: create_ec_pool: ceph osd pool set test allow_ec_overwrites true 2026-02-17T21:24:01.871 INFO:tasks.workunit.client.0.trial031.stderr:set pool 1 allow_ec_overwrites to true 2026-02-17T21:24:01.883 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T21:24:01.883 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:24:01.883 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:24:01.883 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:24:01.884 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T21:24:01.884 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:24:01.885 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:24:01.885 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:24:01.885 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:24:01.885 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:24:01.997 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T21:24:01.997 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:24:01.997 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:24:01.997 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:24:01.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:24:01.998 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:24:02.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:24:02.202 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:24:02.202 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:24:02.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:24:02.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:24:02.203 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:24:02.281 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T21:24:02.282 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T21:24:02.282 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T21:24:02.282 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:24:02.282 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:24:02.362 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672965 2026-02-17T21:24:02.362 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672965 2026-02-17T21:24:02.362 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672965' 2026-02-17T21:24:02.363 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:24:02.363 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:24:02.440 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542147 2026-02-17T21:24:02.440 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542147 2026-02-17T21:24:02.440 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672965 2-60129542147' 2026-02-17T21:24:02.440 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:24:02.441 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T21:24:02.441 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:24:02.442 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:24:02.443 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T21:24:02.443 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:24:02.444 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T21:24:02.444 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T21:24:02.445 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 21474836487 2026-02-17T21:24:02.445 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:24:02.650 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T21:24:02.650 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:24:02.650 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672965 2026-02-17T21:24:02.651 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:24:02.652 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:24:02.652 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672965 2026-02-17T21:24:02.652 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:24:02.653 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672965 2026-02-17T21:24:02.654 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 42949672965 2026-02-17T21:24:02.654 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672965' 2026-02-17T21:24:02.654 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:24:02.863 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672965 -lt 42949672965 2026-02-17T21:24:02.863 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:24:02.864 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542147 2026-02-17T21:24:02.864 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:24:02.865 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:24:02.866 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542147 2026-02-17T21:24:02.866 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:24:02.867 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542147 2026-02-17T21:24:02.867 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-17T21:24:02.868 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 60129542147 2026-02-17T21:24:02.868 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:24:03.077 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-17T21:24:03.078 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:24:03.078 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:24:03.078 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:24:03.343 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T21:24:03.343 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:24:03.344 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:24:03.344 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:24:03.344 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:24:03.344 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:24:03.344 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:24:03.344 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:24:03.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T21:24:03.545 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:24:03.546 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:24:03.546 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:24:03.815 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T21:24:03.815 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T21:24:03.815 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T21:24:03.816 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T21:24:03.816 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:248: TEST_divergent_ec: flush_pg_stats 2026-02-17T21:24:03.816 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:24:03.816 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:24:04.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:24:04.016 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:24:04.017 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:24:04.017 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:24:04.017 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:24:04.017 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:24:04.097 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-17T21:24:04.097 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-17T21:24:04.098 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T21:24:04.098 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:24:04.098 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:24:04.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672967 2026-02-17T21:24:04.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672967 2026-02-17T21:24:04.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-42949672967' 2026-02-17T21:24:04.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:24:04.176 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:24:04.257 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542150 2026-02-17T21:24:04.257 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542150 2026-02-17T21:24:04.257 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-42949672967 2-60129542150' 2026-02-17T21:24:04.257 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:24:04.258 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-17T21:24:04.258 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:24:04.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:24:04.260 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-17T21:24:04.260 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:24:04.261 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-17T21:24:04.261 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 21474836489 2026-02-17T21:24:04.262 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T21:24:04.262 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:24:04.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-17T21:24:04.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:24:05.473 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:24:05.474 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:24:05.683 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-17T21:24:05.683 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:24:05.684 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672967 2026-02-17T21:24:05.684 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:24:05.685 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:24:05.686 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672967 2026-02-17T21:24:05.686 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:24:05.687 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672967 2026-02-17T21:24:05.688 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 42949672967 2026-02-17T21:24:05.688 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672967' 2026-02-17T21:24:05.688 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:24:05.897 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672967 -lt 42949672967 2026-02-17T21:24:05.897 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:24:05.898 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542150 2026-02-17T21:24:05.898 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:24:05.899 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:24:05.900 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542150 2026-02-17T21:24:05.900 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:24:05.901 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542150 2026-02-17T21:24:05.901 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 60129542150 2026-02-17T21:24:05.902 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-17T21:24:05.902 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:24:06.112 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-17T21:24:06.112 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:249: TEST_divergent_ec: wait_for_clean 2026-02-17T21:24:06.112 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:24:06.113 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:24:06.113 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:24:06.113 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T21:24:06.114 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:24:06.114 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:24:06.114 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:24:06.114 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:24:06.114 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:24:06.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T21:24:06.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:24:06.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:24:06.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:24:06.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:24:06.220 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:24:06.430 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:24:06.430 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:24:06.430 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:24:06.430 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:24:06.430 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:24:06.431 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:24:06.509 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836492 2026-02-17T21:24:06.509 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836492 2026-02-17T21:24:06.509 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492' 2026-02-17T21:24:06.510 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:24:06.510 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:24:06.589 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672970 2026-02-17T21:24:06.589 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672970 2026-02-17T21:24:06.590 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-42949672970' 2026-02-17T21:24:06.590 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:24:06.590 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:24:06.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542152 2026-02-17T21:24:06.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542152 2026-02-17T21:24:06.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-42949672970 2-60129542152' 2026-02-17T21:24:06.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:24:06.669 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836492 2026-02-17T21:24:06.669 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:24:06.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:24:06.670 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836492 2026-02-17T21:24:06.671 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:24:06.672 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836492 2026-02-17T21:24:06.672 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-17T21:24:06.672 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 21474836492 2026-02-17T21:24:06.673 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:24:06.873 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836492 2026-02-17T21:24:06.873 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:24:07.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:24:07.875 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:24:08.073 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836492 2026-02-17T21:24:08.073 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:24:09.074 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T21:24:09.075 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:24:09.280 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836492 2026-02-17T21:24:09.281 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:24:09.281 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672970 2026-02-17T21:24:09.281 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:24:09.282 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:24:09.283 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672970 2026-02-17T21:24:09.283 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:24:09.284 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672970 2026-02-17T21:24:09.285 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 42949672970 2026-02-17T21:24:09.285 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-17T21:24:09.285 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:24:09.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672970 -lt 42949672970 2026-02-17T21:24:09.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:24:09.502 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542152 2026-02-17T21:24:09.503 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:24:09.504 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:24:09.504 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542152 2026-02-17T21:24:09.505 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:24:09.506 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542152 2026-02-17T21:24:09.506 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542152' 2026-02-17T21:24:09.506 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 60129542152 2026-02-17T21:24:09.507 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:24:09.706 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542152 -lt 60129542152 2026-02-17T21:24:09.707 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:24:09.708 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:24:09.708 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:24:09.963 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T21:24:09.963 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:24:09.963 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:24:09.963 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:24:09.963 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:24:09.963 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:24:09.964 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:24:09.964 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:24:10.158 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T21:24:10.159 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:24:10.159 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:24:10.159 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:24:10.423 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T21:24:10.423 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T21:24:10.423 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T21:24:10.424 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:252: TEST_divergent_ec: ceph pg dump pgs --format=json 2026-02-17T21:24:10.424 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:252: TEST_divergent_ec: jq '.pg_stats[0].up_primary' 2026-02-17T21:24:10.614 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:24:10.623 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:252: TEST_divergent_ec: local divergent=1 2026-02-17T21:24:10.623 INFO:tasks.workunit.client.0.trial031.stdout:primary and soon to be divergent is 1 2026-02-17T21:24:10.624 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:253: TEST_divergent_ec: echo 'primary and soon to be divergent is 1' 2026-02-17T21:24:10.624 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:254: TEST_divergent_ec: ceph pg dump pgs 2026-02-17T21:24:10.819 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:24:10.819 INFO:tasks.workunit.client.0.trial031.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T21:24:00.999452+0000 0'0 27:17 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:24:00.509416+0000 0'0 2026-02-17T21:24:00.509416+0000 0 0 periodic scrub scheduled @ 2026-02-18T21:53:19.340694+0000 0 0 2026-02-17T21:24:10.819 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:24:10.819 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:24:10.819 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:24:10.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:255: TEST_divergent_ec: local non_divergent= 2026-02-17T21:24:10.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:256: TEST_divergent_ec: for i in $osds 2026-02-17T21:24:10.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:258: TEST_divergent_ec: '[' 0 = 1 ']' 2026-02-17T21:24:10.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:261: TEST_divergent_ec: non_divergent=' 0' 2026-02-17T21:24:10.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:256: TEST_divergent_ec: for i in $osds 2026-02-17T21:24:10.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:258: TEST_divergent_ec: '[' 1 = 1 ']' 2026-02-17T21:24:10.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:259: TEST_divergent_ec: continue 2026-02-17T21:24:10.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:256: TEST_divergent_ec: for i in $osds 2026-02-17T21:24:10.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:258: TEST_divergent_ec: '[' 2 = 1 ']' 2026-02-17T21:24:10.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:261: TEST_divergent_ec: non_divergent=' 0 2' 2026-02-17T21:24:10.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:264: TEST_divergent_ec: echo 'writing initial objects' 2026-02-17T21:24:10.829 INFO:tasks.workunit.client.0.trial031.stdout:writing initial objects 2026-02-17T21:24:10.830 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: seq 1 100 2026-02-17T21:24:10.830 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:10.830 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_1 /etc/fstab 2026-02-17T21:24:10.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:10.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_2 /etc/fstab 2026-02-17T21:24:10.879 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:10.879 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_3 /etc/fstab 2026-02-17T21:24:10.906 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:10.907 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_4 /etc/fstab 2026-02-17T21:24:10.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:10.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_5 /etc/fstab 2026-02-17T21:24:10.964 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:10.964 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_6 /etc/fstab 2026-02-17T21:24:10.992 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:10.992 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_7 /etc/fstab 2026-02-17T21:24:11.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_8 /etc/fstab 2026-02-17T21:24:11.044 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.044 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_9 /etc/fstab 2026-02-17T21:24:11.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_10 /etc/fstab 2026-02-17T21:24:11.100 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.100 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_11 /etc/fstab 2026-02-17T21:24:11.119 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.119 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_12 /etc/fstab 2026-02-17T21:24:11.140 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.140 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_13 /etc/fstab 2026-02-17T21:24:11.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_14 /etc/fstab 2026-02-17T21:24:11.193 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.193 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_15 /etc/fstab 2026-02-17T21:24:11.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_16 /etc/fstab 2026-02-17T21:24:11.245 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.245 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_17 /etc/fstab 2026-02-17T21:24:11.273 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.273 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_18 /etc/fstab 2026-02-17T21:24:11.301 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.301 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_19 /etc/fstab 2026-02-17T21:24:11.328 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.328 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_20 /etc/fstab 2026-02-17T21:24:11.348 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.349 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_21 /etc/fstab 2026-02-17T21:24:11.376 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.376 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_22 /etc/fstab 2026-02-17T21:24:11.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_23 /etc/fstab 2026-02-17T21:24:11.431 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.431 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_24 /etc/fstab 2026-02-17T21:24:11.459 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.459 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_25 /etc/fstab 2026-02-17T21:24:11.488 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.488 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_26 /etc/fstab 2026-02-17T21:24:11.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_27 /etc/fstab 2026-02-17T21:24:11.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_28 /etc/fstab 2026-02-17T21:24:11.572 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.572 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_29 /etc/fstab 2026-02-17T21:24:11.600 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.601 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_30 /etc/fstab 2026-02-17T21:24:11.629 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.629 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_31 /etc/fstab 2026-02-17T21:24:11.657 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.657 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_32 /etc/fstab 2026-02-17T21:24:11.685 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.685 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_33 /etc/fstab 2026-02-17T21:24:11.713 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.713 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_34 /etc/fstab 2026-02-17T21:24:11.741 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.741 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_35 /etc/fstab 2026-02-17T21:24:11.769 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.769 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_36 /etc/fstab 2026-02-17T21:24:11.797 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.797 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_37 /etc/fstab 2026-02-17T21:24:11.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.825 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_38 /etc/fstab 2026-02-17T21:24:11.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_39 /etc/fstab 2026-02-17T21:24:11.880 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.881 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_40 /etc/fstab 2026-02-17T21:24:11.906 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.907 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_41 /etc/fstab 2026-02-17T21:24:11.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_42 /etc/fstab 2026-02-17T21:24:11.955 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.955 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_43 /etc/fstab 2026-02-17T21:24:11.982 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:11.982 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_44 /etc/fstab 2026-02-17T21:24:12.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_45 /etc/fstab 2026-02-17T21:24:12.029 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.029 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_46 /etc/fstab 2026-02-17T21:24:12.049 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.049 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_47 /etc/fstab 2026-02-17T21:24:12.069 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.070 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_48 /etc/fstab 2026-02-17T21:24:12.096 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.097 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_49 /etc/fstab 2026-02-17T21:24:12.122 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.122 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_50 /etc/fstab 2026-02-17T21:24:12.149 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.149 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_51 /etc/fstab 2026-02-17T21:24:12.177 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.177 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_52 /etc/fstab 2026-02-17T21:24:12.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_53 /etc/fstab 2026-02-17T21:24:12.233 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.233 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_54 /etc/fstab 2026-02-17T21:24:12.261 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.261 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_55 /etc/fstab 2026-02-17T21:24:12.280 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.280 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_56 /etc/fstab 2026-02-17T21:24:12.305 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.305 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_57 /etc/fstab 2026-02-17T21:24:12.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_58 /etc/fstab 2026-02-17T21:24:12.360 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.360 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_59 /etc/fstab 2026-02-17T21:24:12.388 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.388 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_60 /etc/fstab 2026-02-17T21:24:12.416 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.416 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_61 /etc/fstab 2026-02-17T21:24:12.444 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.444 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_62 /etc/fstab 2026-02-17T21:24:12.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_63 /etc/fstab 2026-02-17T21:24:12.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_64 /etc/fstab 2026-02-17T21:24:12.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_65 /etc/fstab 2026-02-17T21:24:12.549 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.549 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_66 /etc/fstab 2026-02-17T21:24:12.577 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.577 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_67 /etc/fstab 2026-02-17T21:24:12.605 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.606 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_68 /etc/fstab 2026-02-17T21:24:12.626 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.626 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_69 /etc/fstab 2026-02-17T21:24:12.646 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.646 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_70 /etc/fstab 2026-02-17T21:24:12.666 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.666 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_71 /etc/fstab 2026-02-17T21:24:12.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_72 /etc/fstab 2026-02-17T21:24:12.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_73 /etc/fstab 2026-02-17T21:24:12.751 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.752 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_74 /etc/fstab 2026-02-17T21:24:12.780 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.780 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_75 /etc/fstab 2026-02-17T21:24:12.808 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.808 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_76 /etc/fstab 2026-02-17T21:24:12.836 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.836 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_77 /etc/fstab 2026-02-17T21:24:12.864 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.864 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_78 /etc/fstab 2026-02-17T21:24:12.892 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.892 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_79 /etc/fstab 2026-02-17T21:24:12.920 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.920 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_80 /etc/fstab 2026-02-17T21:24:12.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_81 /etc/fstab 2026-02-17T21:24:12.977 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:12.977 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_82 /etc/fstab 2026-02-17T21:24:13.005 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.005 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_83 /etc/fstab 2026-02-17T21:24:13.032 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.033 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_84 /etc/fstab 2026-02-17T21:24:13.060 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.060 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_85 /etc/fstab 2026-02-17T21:24:13.089 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.089 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_86 /etc/fstab 2026-02-17T21:24:13.117 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.117 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_87 /etc/fstab 2026-02-17T21:24:13.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_88 /etc/fstab 2026-02-17T21:24:13.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.173 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_89 /etc/fstab 2026-02-17T21:24:13.191 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.192 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_90 /etc/fstab 2026-02-17T21:24:13.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_91 /etc/fstab 2026-02-17T21:24:13.245 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.245 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_92 /etc/fstab 2026-02-17T21:24:13.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_93 /etc/fstab 2026-02-17T21:24:13.301 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.301 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_94 /etc/fstab 2026-02-17T21:24:13.329 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.330 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_95 /etc/fstab 2026-02-17T21:24:13.357 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.357 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_96 /etc/fstab 2026-02-17T21:24:13.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_97 /etc/fstab 2026-02-17T21:24:13.413 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.413 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_98 /etc/fstab 2026-02-17T21:24:13.433 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.433 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_99 /etc/fstab 2026-02-17T21:24:13.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-17T21:24:13.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_100 /etc/fstab 2026-02-17T21:24:13.489 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:271: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-17T21:24:13.489 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:271: TEST_divergent_ec: wait_for_clean 2026-02-17T21:24:13.489 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:24:13.489 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:24:13.489 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:24:13.490 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-17T21:24:13.490 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:24:13.490 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:24:13.490 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:24:13.491 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:24:13.491 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:24:13.553 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-17T21:24:13.553 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:24:13.553 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:24:13.553 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:24:13.553 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:24:13.553 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:24:13.759 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:24:13.759 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:24:13.760 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:24:13.760 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:24:13.760 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:24:13.760 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:24:13.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836495 2026-02-17T21:24:13.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836495 2026-02-17T21:24:13.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836495' 2026-02-17T21:24:13.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:24:13.838 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:24:13.920 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672973 2026-02-17T21:24:13.920 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672973 2026-02-17T21:24:13.921 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836495 1-42949672973' 2026-02-17T21:24:13.921 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:24:13.921 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:24:13.999 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542156 2026-02-17T21:24:13.999 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542156 2026-02-17T21:24:14.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836495 1-42949672973 2-60129542156' 2026-02-17T21:24:14.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:24:14.000 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836495 2026-02-17T21:24:14.000 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:24:14.001 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:24:14.002 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836495 2026-02-17T21:24:14.002 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:24:14.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836495 2026-02-17T21:24:14.004 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 21474836495 2026-02-17T21:24:14.004 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836495' 2026-02-17T21:24:14.004 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:24:14.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836495 2026-02-17T21:24:14.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:24:15.203 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:24:15.203 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:24:15.413 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836495 -lt 21474836495 2026-02-17T21:24:15.414 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:24:15.414 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672973 2026-02-17T21:24:15.414 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:24:15.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:24:15.416 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672973 2026-02-17T21:24:15.416 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:24:15.418 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672973 2026-02-17T21:24:15.418 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672973' 2026-02-17T21:24:15.418 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 42949672973 2026-02-17T21:24:15.419 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:24:15.625 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672973 -lt 42949672973 2026-02-17T21:24:15.625 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:24:15.626 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542156 2026-02-17T21:24:15.626 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:24:15.627 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:24:15.628 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542156 2026-02-17T21:24:15.628 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:24:15.629 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542156 2026-02-17T21:24:15.629 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 60129542156 2026-02-17T21:24:15.630 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542156' 2026-02-17T21:24:15.630 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:24:15.844 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542156 -lt 60129542156 2026-02-17T21:24:15.845 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:24:15.845 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:24:15.846 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:24:16.112 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T21:24:16.112 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:24:16.113 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:24:16.113 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:24:16.113 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:24:16.114 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:24:16.114 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:24:16.114 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:24:16.319 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T21:24:16.319 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:24:16.320 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:24:16.320 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:24:16.587 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T21:24:16.587 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T21:24:16.587 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T21:24:16.587 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:273: TEST_divergent_ec: get_pg test existing_1 2026-02-17T21:24:16.587 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-17T21:24:16.587 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=existing_1 2026-02-17T21:24:16.588 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test existing_1 2026-02-17T21:24:16.588 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T21:24:16.796 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:273: TEST_divergent_ec: local pgid=1.0 2026-02-17T21:24:16.796 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:276: TEST_divergent_ec: echo 'blackholing osds 0 2' 2026-02-17T21:24:16.797 INFO:tasks.workunit.client.0.trial031.stdout:blackholing osds 0 2 2026-02-17T21:24:16.797 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:277: TEST_divergent_ec: ceph pg dump pgs 2026-02-17T21:24:16.982 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:24:16.983 INFO:tasks.workunit.client.0.trial031.stdout:1.0 100 0 0 0 0 70600 0 0 16 84 16 active+clean 2026-02-17T21:24:00.999452+0000 27'100 27:117 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:24:00.509416+0000 0'0 2026-02-17T21:24:00.509416+0000 0 0 periodic scrub scheduled @ 2026-02-18T21:53:19.340694+0000 0 0 2026-02-17T21:24:16.983 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:24:16.983 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:24:16.983 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:24:16.990 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:278: TEST_divergent_ec: for i in $non_divergent 2026-02-17T21:24:16.991 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: get_asok_path osd.0 2026-02-17T21:24:16.991 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T21:24:16.992 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T21:24:16.992 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:24:16.992 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:24:16.992 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:24:16.992 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.0.asok 2026-02-17T21:24:16.993 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: CEPH_ARGS= 2026-02-17T21:24:16.993 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-17T21:24:17.045 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:24:17.045 INFO:tasks.workunit.client.0.trial031.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-17T21:24:17.045 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:24:17.050 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:278: TEST_divergent_ec: for i in $non_divergent 2026-02-17T21:24:17.051 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: get_asok_path osd.2 2026-02-17T21:24:17.051 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-17T21:24:17.051 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-17T21:24:17.052 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:24:17.052 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:24:17.052 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:24:17.052 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.2.asok 2026-02-17T21:24:17.052 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: CEPH_ARGS= 2026-02-17T21:24:17.052 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-17T21:24:17.102 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:24:17.102 INFO:tasks.workunit.client.0.trial031.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-17T21:24:17.102 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:24:17.108 INFO:tasks.workunit.client.0.trial031.stdout:writing divergent object 2026-02-17T21:24:17.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:284: TEST_divergent_ec: echo 'writing divergent object' 2026-02-17T21:24:17.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:286: TEST_divergent_ec: sleep 1 2026-02-17T21:24:17.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:285: TEST_divergent_ec: rados -p test put existing_100 /etc/resolv.conf 2026-02-17T21:24:18.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:288: TEST_divergent_ec: rados -p test mksnap snap1 2026-02-17T21:24:18.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:287: TEST_divergent_ec: rados -p test put existing_100 /etc/fstab 2026-02-17T21:24:18.181 INFO:tasks.workunit.client.0.trial031.stdout:created pool test snap snap1 2026-02-17T21:24:18.183 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:290: TEST_divergent_ec: sleep 10 2026-02-17T21:24:18.184 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:289: TEST_divergent_ec: expr 100 - 1 2026-02-17T21:24:18.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:289: TEST_divergent_ec: rados -p test put existing_99 /etc/fstab 2026-02-17T21:24:28.184 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:291: TEST_divergent_ec: killall -9 rados 2026-02-17T21:24:28.226 INFO:tasks.workunit.client.0.trial031.stderr:rados: no process found 2026-02-17T21:24:28.226 INFO:tasks.workunit.client.0.trial031.stdout:killing all the osds 2026-02-17T21:24:28.227 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:294: TEST_divergent_ec: echo 'killing all the osds' 2026-02-17T21:24:28.227 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:295: TEST_divergent_ec: ceph pg dump pgs 2026-02-17T21:24:28.428 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:24:28.428 INFO:tasks.workunit.client.0.trial031.stdout:1.0 101 0 0 0 0 71306 0 0 14 90 14 active+clean 2026-02-17T21:24:00.999452+0000 28'104 28:123 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:24:00.509416+0000 0'0 2026-02-17T21:24:00.509416+0000 0 0 periodic scrub scheduled @ 2026-02-19T08:28:51.749088+0000 0 0 2026-02-17T21:24:28.429 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:24:28.429 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:24:28.429 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:24:28.436 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:296: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd 2026-02-17T21:24:28.437 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:24:28.437 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:24:28.437 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:24:28.438 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:24:28.438 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:24:28.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:24:28.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:297: TEST_divergent_ec: for i in $osds 2026-02-17T21:24:28.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:299: TEST_divergent_ec: ceph osd down osd.0 2026-02-17T21:24:28.803 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already down. 2026-02-17T21:24:28.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:297: TEST_divergent_ec: for i in $osds 2026-02-17T21:24:28.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:299: TEST_divergent_ec: ceph osd down osd.1 2026-02-17T21:24:29.080 INFO:tasks.workunit.client.0.trial031.stderr:osd.1 is already down. 2026-02-17T21:24:29.090 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:297: TEST_divergent_ec: for i in $osds 2026-02-17T21:24:29.090 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:299: TEST_divergent_ec: ceph osd down osd.2 2026-02-17T21:24:29.360 INFO:tasks.workunit.client.0.trial031.stderr:osd.2 is already down. 2026-02-17T21:24:29.369 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:301: TEST_divergent_ec: for i in $non_divergent 2026-02-17T21:24:29.370 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:303: TEST_divergent_ec: ceph osd out osd.0 2026-02-17T21:24:29.637 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already out. 2026-02-17T21:24:29.646 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:301: TEST_divergent_ec: for i in $non_divergent 2026-02-17T21:24:29.647 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:303: TEST_divergent_ec: ceph osd out osd.2 2026-02-17T21:24:29.909 INFO:tasks.workunit.client.0.trial031.stderr:osd.2 is already out. 2026-02-17T21:24:29.920 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:307: TEST_divergent_ec: echo 'bringing up non_divergent 0 2' 2026-02-17T21:24:29.921 INFO:tasks.workunit.client.0.trial031.stdout:bringing up non_divergent 0 2 2026-02-17T21:24:29.921 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:308: TEST_divergent_ec: ceph pg dump pgs 2026-02-17T21:24:30.114 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:24:30.114 INFO:tasks.workunit.client.0.trial031.stdout:1.0 101 0 0 0 0 71306 0 0 14 90 14 stale+active+clean 2026-02-17T21:24:00.999452+0000 28'104 28:123 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:24:00.509416+0000 0'0 2026-02-17T21:24:00.509416+0000 0 0 periodic scrub scheduled @ 2026-02-19T08:28:51.749088+0000 0 0 2026-02-17T21:24:30.115 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:24:30.115 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:24:30.115 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:24:30.126 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:309: TEST_divergent_ec: for i in $non_divergent 2026-02-17T21:24:30.126 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:311: TEST_divergent_ec: activate_osd td/divergent-priors 0 2026-02-17T21:24:30.126 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:24:30.126 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:24:30.126 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T21:24:30.127 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:24:30.127 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/0 2026-02-17T21:24:30.127 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:24:30.127 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:24:30.127 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:24:30.127 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:24:30.127 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-17T21:24:30.127 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-17T21:24:30.127 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:24:30.127 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:24:30.127 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:24:30.127 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:24:30.128 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:24:30.128 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:24:30.128 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:24:30.128 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:24:30.128 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:24:30.128 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:24:30.128 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:24:30.128 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:24:30.128 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:24:30.128 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:24:30.128 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:24:30.128 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:24:30.128 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:24:30.129 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:24:30.129 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:24:30.129 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:24:30.129 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/0 2026-02-17T21:24:30.129 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T21:24:30.130 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:24:30.130 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:24:30.131 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/0/whoami 2026-02-17T21:24:30.131 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T21:24:30.131 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:24:30.131 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:24:30.131 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:24:30.144 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:24:30.142+0000 7f0238f328c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:24:30.146 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:24:30.142+0000 7f0238f328c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:24:30.147 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:24:30.142+0000 7f0238f328c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:24:30.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T21:24:30.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:24:30.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:24:30.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:24:30.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:24:30.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:24:30.338 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:24:30.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:24:30.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:24:30.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:24:30.542 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:24:31.429 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:24:31.426+0000 7f0238f328c0 -1 Falling back to public interface 2026-02-17T21:24:31.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:24:31.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:24:31.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:24:31.545 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:24:31.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:24:31.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:24:31.754 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:24:32.756 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:24:32.756 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:24:32.756 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:24:32.756 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:24:32.756 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:24:32.756 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:24:32.956 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:24:33.958 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:24:33.958 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:24:33.958 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:24:33.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:24:33.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:24:33.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:24:34.161 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:24:34.705 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:24:34.702+0000 7f0238f328c0 -1 osd.0 27 log_to_monitors true 2026-02-17T21:24:35.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:24:35.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:24:35.162 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:24:35.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:24:35.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:24:35.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:24:35.369 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up out weight 0 up_from 35 up_thru 0 down_at 29 last_clean_interval [5,27) [v2:127.0.0.1:6802/2872564052,v1:127.0.0.1:6803/2872564052] [v2:127.0.0.1:6804/2872564052,v1:127.0.0.1:6805/2872564052] exists,up 27052348-33c5-4b28-9883-30bfd1415320 2026-02-17T21:24:35.372 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:24:35.372 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:24:35.373 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:24:35.373 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:309: TEST_divergent_ec: for i in $non_divergent 2026-02-17T21:24:35.373 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:311: TEST_divergent_ec: activate_osd td/divergent-priors 2 2026-02-17T21:24:35.373 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:24:35.373 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:24:35.373 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T21:24:35.373 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:24:35.373 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/2 2026-02-17T21:24:35.373 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:24:35.373 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:24:35.373 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:24:35.374 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:24:35.374 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-17T21:24:35.374 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-17T21:24:35.374 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:24:35.374 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:24:35.374 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:24:35.374 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:24:35.374 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:24:35.374 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:24:35.374 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:24:35.374 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:24:35.374 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:24:35.374 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:24:35.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:24:35.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:24:35.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:24:35.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:24:35.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:24:35.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:24:35.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:24:35.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:24:35.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:24:35.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:24:35.375 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/2 2026-02-17T21:24:35.376 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:24:35.377 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T21:24:35.377 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:24:35.377 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/2/whoami 2026-02-17T21:24:35.377 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T21:24:35.377 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:24:35.377 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:24:35.377 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:24:35.398 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:24:35.394+0000 7f72b36b28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:24:35.400 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:24:35.398+0000 7f72b36b28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:24:35.401 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:24:35.398+0000 7f72b36b28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:24:35.582 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T21:24:35.582 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:24:35.582 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:24:35.582 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:24:35.583 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:24:35.583 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:24:35.583 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:24:35.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:24:35.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:24:35.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:24:35.789 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:24:36.645 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:24:36.642+0000 7f72b36b28c0 -1 Falling back to public interface 2026-02-17T21:24:36.790 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:24:36.790 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:24:36.790 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:24:36.790 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:24:36.791 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:24:36.791 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:24:36.994 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:24:37.996 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:24:37.996 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:24:37.997 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:24:37.997 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:24:37.997 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:24:37.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:24:38.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:24:39.203 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:24:39.203 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:24:39.204 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:24:39.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:24:39.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:24:39.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:24:39.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:24:39.920 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:24:39.918+0000 7f72b36b28c0 -1 osd.2 27 log_to_monitors true 2026-02-17T21:24:40.405 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:24:40.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:24:40.406 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:24:40.407 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:24:40.407 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:24:40.407 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:24:40.610 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up out weight 0 up_from 37 up_thru 0 down_at 31 last_clean_interval [14,27) [v2:127.0.0.1:6810/3154825036,v1:127.0.0.1:6811/3154825036] [v2:127.0.0.1:6812/3154825036,v1:127.0.0.1:6813/3154825036] exists,up a63bbb72-c70c-48fa-b386-d5e9b16de7b4 2026-02-17T21:24:40.611 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:24:40.611 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:24:40.611 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:24:40.611 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:313: TEST_divergent_ec: for i in $non_divergent 2026-02-17T21:24:40.611 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:315: TEST_divergent_ec: ceph osd in osd.0 2026-02-17T21:24:40.865 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already in. 2026-02-17T21:24:40.875 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:313: TEST_divergent_ec: for i in $non_divergent 2026-02-17T21:24:40.875 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:315: TEST_divergent_ec: ceph osd in osd.2 2026-02-17T21:24:41.168 INFO:tasks.workunit.client.0.trial031.stderr:osd.2 is already in. 2026-02-17T21:24:41.178 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:318: TEST_divergent_ec: sleep 5 2026-02-17T21:24:46.181 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:322: TEST_divergent_ec: expr 2 + 2 2026-02-17T21:24:46.182 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:322: TEST_divergent_ec: objname=existing_4 2026-02-17T21:24:46.182 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:323: TEST_divergent_ec: echo 'writing non-divergent object existing_4' 2026-02-17T21:24:46.182 INFO:tasks.workunit.client.0.trial031.stdout:writing non-divergent object existing_4 2026-02-17T21:24:46.183 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:324: TEST_divergent_ec: ceph pg dump pgs 2026-02-17T21:24:46.370 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:24:46.371 INFO:tasks.workunit.client.0.trial031.stdout:1.0 101 0 0 0 0 71306 0 0 14 90 14 stale+active+clean 2026-02-17T21:24:00.999452+0000 28'104 28:123 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:24:00.509416+0000 0'0 2026-02-17T21:24:00.509416+0000 0 0 periodic scrub scheduled @ 2026-02-19T08:28:51.749088+0000 0 0 2026-02-17T21:24:46.371 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:24:46.371 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:24:46.371 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:24:46.379 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:325: TEST_divergent_ec: rados -p test put existing_4 /etc/resolv.conf 2026-02-17T21:24:46.408 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:327: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-17T21:24:46.408 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:327: TEST_divergent_ec: wait_for_clean 2026-02-17T21:24:46.408 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:24:46.408 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:24:46.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:24:46.409 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-17T21:24:46.410 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:24:46.410 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:24:46.410 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:24:46.410 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:24:46.410 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:24:46.479 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-17T21:24:46.480 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:24:46.480 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:24:46.480 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:24:46.480 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:24:46.480 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:24:46.687 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:24:46.687 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:24:46.687 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:24:46.688 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:24:46.688 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:24:46.688 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:24:46.766 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855365 2026-02-17T21:24:46.766 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855365 2026-02-17T21:24:46.766 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855365' 2026-02-17T21:24:46.766 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:24:46.766 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:24:46.826 INFO:tasks.workunit.client.0.trial031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-17T21:24:46.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-17T21:24:46.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-17T21:24:46.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-17T21:24:46.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:24:46.829 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:24:46.908 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789956 2026-02-17T21:24:46.908 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789956 2026-02-17T21:24:46.908 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855365 2-158913789956' 2026-02-17T21:24:46.908 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:24:46.909 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855365 2026-02-17T21:24:46.909 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:24:46.910 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:24:46.910 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855365 2026-02-17T21:24:46.911 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:24:46.912 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855365 2026-02-17T21:24:46.912 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 150323855365 2026-02-17T21:24:46.913 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855365' 2026-02-17T21:24:46.913 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:24:47.112 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855363 -lt 150323855365 2026-02-17T21:24:47.113 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:24:48.114 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:24:48.114 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:24:48.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855363 -lt 150323855365 2026-02-17T21:24:48.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:24:49.318 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T21:24:49.319 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:24:49.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855365 -lt 150323855365 2026-02-17T21:24:49.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:24:49.522 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-158913789956 2026-02-17T21:24:49.522 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:24:49.523 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:24:49.524 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-158913789956 2026-02-17T21:24:49.524 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:24:49.525 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789956 2026-02-17T21:24:49.526 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 158913789956 2026-02-17T21:24:49.526 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 158913789956' 2026-02-17T21:24:49.526 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:24:49.743 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789956 -lt 158913789956 2026-02-17T21:24:49.744 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:24:49.744 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:24:49.744 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:24:50.024 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T21:24:50.024 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:24:50.024 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:24:50.024 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:24:50.025 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:24:50.025 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:24:50.025 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:24:50.025 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:24:50.231 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:24:50.232 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:24:50.232 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:24:50.232 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:24:50.501 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:24:50.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T21:24:50.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T21:24:50.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T21:24:50.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:24:50.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T21:24:50.602 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:24:50.602 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:24:50.603 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:24:50.603 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:24:50.604 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:24:50.604 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:24:50.604 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:24:50.604 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:24:50.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:24:50.811 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:24:50.811 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:24:50.811 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:24:51.082 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:24:51.082 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:24:51.082 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:24:51.082 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:24:51.082 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:24:51.082 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:24:51.082 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:24:51.082 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:24:51.082 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:24:51.358 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:24:51.358 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:24:51.358 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-17T21:24:51.358 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:24:51.358 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T21:24:51.558 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:24:51.559 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:24:51.560 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:24:51.560 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:24:51.560 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:24:51.560 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:24:51.560 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:24:51.560 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:24:51.765 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:24:51.765 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:24:51.766 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:24:51.766 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:24:52.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:24:52.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:24:52.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:24:52.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:24:52.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:24:52.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:24:52.037 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:24:52.038 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:24:52.038 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:24:52.308 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:24:52.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:24:52.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-17T21:24:52.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:24:52.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T21:24:52.709 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:24:52.709 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:24:52.710 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:24:52.711 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:24:52.711 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:24:52.711 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:24:52.711 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:24:52.711 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:24:52.912 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:24:52.912 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:24:52.913 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:24:52.913 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:24:53.179 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:24:53.179 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:24:53.179 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:24:53.179 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:24:53.179 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:24:53.179 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:24:53.179 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:24:53.179 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:24:53.180 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:24:53.447 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:24:53.447 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:24:53.447 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-17T21:24:53.447 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:24:53.448 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T21:24:54.247 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:24:54.247 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:24:54.248 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:24:54.248 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:24:54.248 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:24:54.248 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:24:54.248 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:24:54.248 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:24:54.453 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:24:54.454 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:24:54.455 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:24:54.455 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:24:54.725 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:24:54.725 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:24:54.725 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:24:54.725 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:24:54.725 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:24:54.725 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:24:54.725 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:24:54.725 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:24:54.726 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:24:54.985 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:24:54.986 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:24:54.986 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-17T21:24:54.986 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:24:54.986 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-17T21:24:56.587 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:24:56.587 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:24:56.588 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:24:56.588 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:24:56.588 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:24:56.588 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:24:56.588 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:24:56.588 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:24:56.783 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:24:56.784 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:24:56.784 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:24:56.784 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:24:57.052 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:24:57.052 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:24:57.053 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:24:57.053 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:24:57.053 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:24:57.053 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:24:57.053 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:24:57.053 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:24:57.054 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:24:57.322 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:24:57.322 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:24:57.322 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-17T21:24:57.322 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:24:57.322 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-17T21:25:00.523 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:25:00.523 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:25:00.524 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:25:00.524 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:25:00.524 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:25:00.524 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:25:00.524 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:25:00.524 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:25:00.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:25:00.729 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:25:00.729 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:25:00.729 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:25:00.994 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:25:00.995 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:25:00.995 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:25:00.995 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:25:00.995 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:25:00.995 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:25:00.995 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:25:00.995 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:25:00.995 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:25:01.265 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:25:01.266 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:25:01.266 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-17T21:25:01.266 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:25:01.266 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-17T21:25:07.666 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:25:07.666 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:25:07.666 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:25:07.667 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:25:07.667 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:25:07.667 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:25:07.667 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:25:07.667 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:25:07.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:25:07.869 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:25:07.870 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:25:07.870 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:25:08.141 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:25:08.141 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:25:08.141 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:25:08.142 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:25:08.142 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:25:08.142 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:25:08.142 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:25:08.142 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:25:08.142 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:25:08.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:25:08.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:25:08.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-17T21:25:08.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:25:08.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-17T21:25:15.710 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:25:15.710 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:25:15.711 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:25:15.711 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:25:15.711 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:25:15.712 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:25:15.712 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:25:15.712 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:25:15.912 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:25:15.913 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:25:15.913 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:25:15.914 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:25:16.181 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:25:16.181 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:25:16.181 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:25:16.182 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:25:16.182 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:25:16.182 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:25:16.182 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:25:16.182 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:25:16.182 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:25:16.450 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:25:16.450 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:25:16.450 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-17T21:25:16.450 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-17T21:25:16.703 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:25:16.704 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_fingerprint": "a569169b-2a17-4b91-b9bb-24e9547abd0e", 2026-02-17T21:25:16.704 INFO:tasks.workunit.client.0.trial031.stdout: "version": "19.2.3-679-gd855f53b", 2026-02-17T21:25:16.704 INFO:tasks.workunit.client.0.trial031.stdout: "commit": "d855f53b89fdcec760fd9232a5fb55ed4fb111a1", 2026-02-17T21:25:16.704 INFO:tasks.workunit.client.0.trial031.stdout: "timestamp": "2026-02-17T21:25:16.703635+0000", 2026-02-17T21:25:16.704 INFO:tasks.workunit.client.0.trial031.stdout: "tag": "", 2026-02-17T21:25:16.704 INFO:tasks.workunit.client.0.trial031.stdout: "health": { 2026-02-17T21:25:16.704 INFO:tasks.workunit.client.0.trial031.stdout: "status": "HEALTH_WARN", 2026-02-17T21:25:16.704 INFO:tasks.workunit.client.0.trial031.stdout: "checks": { 2026-02-17T21:25:16.704 INFO:tasks.workunit.client.0.trial031.stdout: "OSDMAP_FLAGS": { 2026-02-17T21:25:16.704 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:25:16.704 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:25:16.705 INFO:tasks.workunit.client.0.trial031.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-17T21:25:16.705 INFO:tasks.workunit.client.0.trial031.stdout: "count": 17 2026-02-17T21:25:16.705 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.705 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [], 2026-02-17T21:25:16.705 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:25:16.705 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.705 INFO:tasks.workunit.client.0.trial031.stdout: "OSD_DOWN": { 2026-02-17T21:25:16.705 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:25:16.705 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:25:16.705 INFO:tasks.workunit.client.0.trial031.stdout: "message": "1 osds down", 2026-02-17T21:25:16.705 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:25:16.705 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.706 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:25:16.706 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.706 INFO:tasks.workunit.client.0.trial031.stdout: "message": "osd.1 (root=default,host=trial031) is down" 2026-02-17T21:25:16.706 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.706 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.706 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:25:16.706 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.706 INFO:tasks.workunit.client.0.trial031.stdout: "PG_DEGRADED": { 2026-02-17T21:25:16.706 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:25:16.706 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:25:16.706 INFO:tasks.workunit.client.0.trial031.stdout: "message": "Degraded data redundancy: 101/303 objects degraded (33.333%), 1 pg degraded", 2026-02-17T21:25:16.707 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:25:16.707 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.707 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:25:16.707 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.707 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [NONE,0,2]" 2026-02-17T21:25:16.707 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.707 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.707 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:25:16.707 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.707 INFO:tasks.workunit.client.0.trial031.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-17T21:25:16.707 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:25:16.707 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:25:16.708 INFO:tasks.workunit.client.0.trial031.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-17T21:25:16.708 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:25:16.708 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.708 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:25:16.708 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.708 INFO:tasks.workunit.client.0.trial031.stdout: "message": "application not enabled on pool 'test'" 2026-02-17T21:25:16.708 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.708 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.708 INFO:tasks.workunit.client.0.trial031.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-17T21:25:16.708 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.709 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.709 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:25:16.709 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.709 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.709 INFO:tasks.workunit.client.0.trial031.stdout: "mutes": [] 2026-02-17T21:25:16.709 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.709 INFO:tasks.workunit.client.0.trial031.stdout: "monmap_first_committed": 1, 2026-02-17T21:25:16.709 INFO:tasks.workunit.client.0.trial031.stdout: "monmap_last_committed": 1, 2026-02-17T21:25:16.709 INFO:tasks.workunit.client.0.trial031.stdout: "monmap": { 2026-02-17T21:25:16.709 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 1, 2026-02-17T21:25:16.709 INFO:tasks.workunit.client.0.trial031.stdout: "fsid": "5ebac6e7-5321-4893-ad68-c06127ed455b", 2026-02-17T21:25:16.709 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:23:31.926448Z", 2026-02-17T21:25:16.710 INFO:tasks.workunit.client.0.trial031.stdout: "created": "2026-02-17T21:23:31.926448Z", 2026-02-17T21:25:16.710 INFO:tasks.workunit.client.0.trial031.stdout: "min_mon_release": 19, 2026-02-17T21:25:16.710 INFO:tasks.workunit.client.0.trial031.stdout: "min_mon_release_name": "squid", 2026-02-17T21:25:16.710 INFO:tasks.workunit.client.0.trial031.stdout: "election_strategy": 1, 2026-02-17T21:25:16.710 INFO:tasks.workunit.client.0.trial031.stdout: "disallowed_leaders": "", 2026-02-17T21:25:16.710 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode": false, 2026-02-17T21:25:16.710 INFO:tasks.workunit.client.0.trial031.stdout: "tiebreaker_mon": "", 2026-02-17T21:25:16.710 INFO:tasks.workunit.client.0.trial031.stdout: "removed_ranks": "", 2026-02-17T21:25:16.710 INFO:tasks.workunit.client.0.trial031.stdout: "features": { 2026-02-17T21:25:16.710 INFO:tasks.workunit.client.0.trial031.stdout: "persistent": [ 2026-02-17T21:25:16.710 INFO:tasks.workunit.client.0.trial031.stdout: "kraken", 2026-02-17T21:25:16.710 INFO:tasks.workunit.client.0.trial031.stdout: "luminous", 2026-02-17T21:25:16.710 INFO:tasks.workunit.client.0.trial031.stdout: "mimic", 2026-02-17T21:25:16.711 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap-prune", 2026-02-17T21:25:16.711 INFO:tasks.workunit.client.0.trial031.stdout: "nautilus", 2026-02-17T21:25:16.711 INFO:tasks.workunit.client.0.trial031.stdout: "octopus", 2026-02-17T21:25:16.711 INFO:tasks.workunit.client.0.trial031.stdout: "pacific", 2026-02-17T21:25:16.711 INFO:tasks.workunit.client.0.trial031.stdout: "elector-pinging", 2026-02-17T21:25:16.711 INFO:tasks.workunit.client.0.trial031.stdout: "quincy", 2026-02-17T21:25:16.711 INFO:tasks.workunit.client.0.trial031.stdout: "reef", 2026-02-17T21:25:16.711 INFO:tasks.workunit.client.0.trial031.stdout: "squid" 2026-02-17T21:25:16.711 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.711 INFO:tasks.workunit.client.0.trial031.stdout: "optional": [] 2026-02-17T21:25:16.711 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.711 INFO:tasks.workunit.client.0.trial031.stdout: "mons": [ 2026-02-17T21:25:16.711 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.712 INFO:tasks.workunit.client.0.trial031.stdout: "rank": 0, 2026-02-17T21:25:16.712 INFO:tasks.workunit.client.0.trial031.stdout: "name": "a", 2026-02-17T21:25:16.712 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:25:16.712 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:16.712 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.712 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:16.712 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:7115", 2026-02-17T21:25:16.712 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 0 2026-02-17T21:25:16.712 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.712 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.712 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.712 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:7115/0", 2026-02-17T21:25:16.713 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-17T21:25:16.713 INFO:tasks.workunit.client.0.trial031.stdout: "priority": 0, 2026-02-17T21:25:16.713 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 0, 2026-02-17T21:25:16.713 INFO:tasks.workunit.client.0.trial031.stdout: "crush_location": "{}" 2026-02-17T21:25:16.713 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.713 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.713 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.713 INFO:tasks.workunit.client.0.trial031.stdout: "quorum": [ 2026-02-17T21:25:16.713 INFO:tasks.workunit.client.0.trial031.stdout: 0 2026-02-17T21:25:16.713 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.713 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap": { 2026-02-17T21:25:16.713 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 41, 2026-02-17T21:25:16.714 INFO:tasks.workunit.client.0.trial031.stdout: "fsid": "5ebac6e7-5321-4893-ad68-c06127ed455b", 2026-02-17T21:25:16.714 INFO:tasks.workunit.client.0.trial031.stdout: "created": "2026-02-17T21:23:32.009422+0000", 2026-02-17T21:25:16.714 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:24:41.218786+0000", 2026-02-17T21:25:16.714 INFO:tasks.workunit.client.0.trial031.stdout: "last_up_change": "2026-02-17T21:24:40.075310+0000", 2026-02-17T21:25:16.714 INFO:tasks.workunit.client.0.trial031.stdout: "last_in_change": "2026-02-17T21:24:41.116674+0000", 2026-02-17T21:25:16.714 INFO:tasks.workunit.client.0.trial031.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-17T21:25:16.714 INFO:tasks.workunit.client.0.trial031.stdout: "flags_num": 5800384, 2026-02-17T21:25:16.714 INFO:tasks.workunit.client.0.trial031.stdout: "flags_set": [ 2026-02-17T21:25:16.714 INFO:tasks.workunit.client.0.trial031.stdout: "nodown", 2026-02-17T21:25:16.714 INFO:tasks.workunit.client.0.trial031.stdout: "noin", 2026-02-17T21:25:16.714 INFO:tasks.workunit.client.0.trial031.stdout: "noout", 2026-02-17T21:25:16.714 INFO:tasks.workunit.client.0.trial031.stdout: "pglog_hardlimit", 2026-02-17T21:25:16.714 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snapdirs", 2026-02-17T21:25:16.715 INFO:tasks.workunit.client.0.trial031.stdout: "recovery_deletes", 2026-02-17T21:25:16.715 INFO:tasks.workunit.client.0.trial031.stdout: "sortbitwise" 2026-02-17T21:25:16.715 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.715 INFO:tasks.workunit.client.0.trial031.stdout: "crush_version": 10, 2026-02-17T21:25:16.715 INFO:tasks.workunit.client.0.trial031.stdout: "full_ratio": 0.99000000953674316, 2026-02-17T21:25:16.715 INFO:tasks.workunit.client.0.trial031.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-17T21:25:16.715 INFO:tasks.workunit.client.0.trial031.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-17T21:25:16.715 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_snapshot": "", 2026-02-17T21:25:16.715 INFO:tasks.workunit.client.0.trial031.stdout: "pool_max": 1, 2026-02-17T21:25:16.715 INFO:tasks.workunit.client.0.trial031.stdout: "max_osd": 3, 2026-02-17T21:25:16.715 INFO:tasks.workunit.client.0.trial031.stdout: "require_min_compat_client": "luminous", 2026-02-17T21:25:16.715 INFO:tasks.workunit.client.0.trial031.stdout: "min_compat_client": "jewel", 2026-02-17T21:25:16.715 INFO:tasks.workunit.client.0.trial031.stdout: "require_osd_release": "squid", 2026-02-17T21:25:16.716 INFO:tasks.workunit.client.0.trial031.stdout: "allow_crimson": false, 2026-02-17T21:25:16.716 INFO:tasks.workunit.client.0.trial031.stdout: "pools": [ 2026-02-17T21:25:16.716 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.716 INFO:tasks.workunit.client.0.trial031.stdout: "pool": 1, 2026-02-17T21:25:16.716 INFO:tasks.workunit.client.0.trial031.stdout: "pool_name": "test", 2026-02-17T21:25:16.716 INFO:tasks.workunit.client.0.trial031.stdout: "create_time": "2026-02-17T21:24:00.416440+0000", 2026-02-17T21:25:16.716 INFO:tasks.workunit.client.0.trial031.stdout: "flags": 16389, 2026-02-17T21:25:16.716 INFO:tasks.workunit.client.0.trial031.stdout: "flags_names": "hashpspool,ec_overwrites,pool_snaps", 2026-02-17T21:25:16.716 INFO:tasks.workunit.client.0.trial031.stdout: "type": 3, 2026-02-17T21:25:16.716 INFO:tasks.workunit.client.0.trial031.stdout: "size": 3, 2026-02-17T21:25:16.716 INFO:tasks.workunit.client.0.trial031.stdout: "min_size": 2, 2026-02-17T21:25:16.716 INFO:tasks.workunit.client.0.trial031.stdout: "crush_rule": 1, 2026-02-17T21:25:16.716 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_count": 0, 2026-02-17T21:25:16.717 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_target": 0, 2026-02-17T21:25:16.717 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_barrier": 0, 2026-02-17T21:25:16.717 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-17T21:25:16.717 INFO:tasks.workunit.client.0.trial031.stdout: "is_stretch_pool": false, 2026-02-17T21:25:16.717 INFO:tasks.workunit.client.0.trial031.stdout: "object_hash": 2, 2026-02-17T21:25:16.717 INFO:tasks.workunit.client.0.trial031.stdout: "pg_autoscale_mode": "off", 2026-02-17T21:25:16.717 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num": 1, 2026-02-17T21:25:16.717 INFO:tasks.workunit.client.0.trial031.stdout: "pg_placement_num": 1, 2026-02-17T21:25:16.717 INFO:tasks.workunit.client.0.trial031.stdout: "pg_placement_num_target": 1, 2026-02-17T21:25:16.717 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num_target": 1, 2026-02-17T21:25:16.717 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num_pending": 1, 2026-02-17T21:25:16.717 INFO:tasks.workunit.client.0.trial031.stdout: "last_pg_merge_meta": { 2026-02-17T21:25:16.717 INFO:tasks.workunit.client.0.trial031.stdout: "source_pgid": "0.0", 2026-02-17T21:25:16.718 INFO:tasks.workunit.client.0.trial031.stdout: "ready_epoch": 0, 2026-02-17T21:25:16.718 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_started": 0, 2026-02-17T21:25:16.718 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_clean": 0, 2026-02-17T21:25:16.718 INFO:tasks.workunit.client.0.trial031.stdout: "source_version": "0'0", 2026-02-17T21:25:16.718 INFO:tasks.workunit.client.0.trial031.stdout: "target_version": "0'0" 2026-02-17T21:25:16.718 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.718 INFO:tasks.workunit.client.0.trial031.stdout: "last_change": "28", 2026-02-17T21:25:16.718 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend": "0", 2026-02-17T21:25:16.718 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-17T21:25:16.718 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend_preluminous": "0", 2026-02-17T21:25:16.718 INFO:tasks.workunit.client.0.trial031.stdout: "auid": 0, 2026-02-17T21:25:16.718 INFO:tasks.workunit.client.0.trial031.stdout: "snap_mode": "pool", 2026-02-17T21:25:16.718 INFO:tasks.workunit.client.0.trial031.stdout: "snap_seq": 1, 2026-02-17T21:25:16.719 INFO:tasks.workunit.client.0.trial031.stdout: "snap_epoch": 28, 2026-02-17T21:25:16.719 INFO:tasks.workunit.client.0.trial031.stdout: "pool_snaps": [ 2026-02-17T21:25:16.719 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.719 INFO:tasks.workunit.client.0.trial031.stdout: "snapid": 1, 2026-02-17T21:25:16.719 INFO:tasks.workunit.client.0.trial031.stdout: "stamp": "2026-02-17T21:24:18.130543+0000", 2026-02-17T21:25:16.719 INFO:tasks.workunit.client.0.trial031.stdout: "name": "snap1" 2026-02-17T21:25:16.719 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.719 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.719 INFO:tasks.workunit.client.0.trial031.stdout: "removed_snaps": "[]", 2026-02-17T21:25:16.719 INFO:tasks.workunit.client.0.trial031.stdout: "quota_max_bytes": 0, 2026-02-17T21:25:16.719 INFO:tasks.workunit.client.0.trial031.stdout: "quota_max_objects": 0, 2026-02-17T21:25:16.720 INFO:tasks.workunit.client.0.trial031.stdout: "tiers": [], 2026-02-17T21:25:16.720 INFO:tasks.workunit.client.0.trial031.stdout: "tier_of": -1, 2026-02-17T21:25:16.720 INFO:tasks.workunit.client.0.trial031.stdout: "read_tier": -1, 2026-02-17T21:25:16.720 INFO:tasks.workunit.client.0.trial031.stdout: "write_tier": -1, 2026-02-17T21:25:16.720 INFO:tasks.workunit.client.0.trial031.stdout: "cache_mode": "none", 2026-02-17T21:25:16.720 INFO:tasks.workunit.client.0.trial031.stdout: "target_max_bytes": 0, 2026-02-17T21:25:16.720 INFO:tasks.workunit.client.0.trial031.stdout: "target_max_objects": 0, 2026-02-17T21:25:16.720 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-17T21:25:16.720 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-17T21:25:16.720 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-17T21:25:16.720 INFO:tasks.workunit.client.0.trial031.stdout: "cache_min_flush_age": 0, 2026-02-17T21:25:16.720 INFO:tasks.workunit.client.0.trial031.stdout: "cache_min_evict_age": 0, 2026-02-17T21:25:16.720 INFO:tasks.workunit.client.0.trial031.stdout: "erasure_code_profile": "myprofile", 2026-02-17T21:25:16.721 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_params": { 2026-02-17T21:25:16.721 INFO:tasks.workunit.client.0.trial031.stdout: "type": "none" 2026-02-17T21:25:16.721 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.721 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_period": 0, 2026-02-17T21:25:16.721 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_count": 0, 2026-02-17T21:25:16.721 INFO:tasks.workunit.client.0.trial031.stdout: "use_gmt_hitset": true, 2026-02-17T21:25:16.721 INFO:tasks.workunit.client.0.trial031.stdout: "min_read_recency_for_promote": 0, 2026-02-17T21:25:16.721 INFO:tasks.workunit.client.0.trial031.stdout: "min_write_recency_for_promote": 0, 2026-02-17T21:25:16.721 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_grade_decay_rate": 0, 2026-02-17T21:25:16.721 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_search_last_n": 0, 2026-02-17T21:25:16.721 INFO:tasks.workunit.client.0.trial031.stdout: "grade_table": [], 2026-02-17T21:25:16.721 INFO:tasks.workunit.client.0.trial031.stdout: "stripe_width": 8192, 2026-02-17T21:25:16.721 INFO:tasks.workunit.client.0.trial031.stdout: "expected_num_objects": 0, 2026-02-17T21:25:16.721 INFO:tasks.workunit.client.0.trial031.stdout: "fast_read": false, 2026-02-17T21:25:16.722 INFO:tasks.workunit.client.0.trial031.stdout: "options": {}, 2026-02-17T21:25:16.722 INFO:tasks.workunit.client.0.trial031.stdout: "application_metadata": {} 2026-02-17T21:25:16.722 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.722 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.722 INFO:tasks.workunit.client.0.trial031.stdout: "osds": [ 2026-02-17T21:25:16.722 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.722 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:25:16.722 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "27052348-33c5-4b28-9883-30bfd1415320", 2026-02-17T21:25:16.722 INFO:tasks.workunit.client.0.trial031.stdout: "up": 1, 2026-02-17T21:25:16.722 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:25:16.722 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:25:16.722 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:25:16.723 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 5, 2026-02-17T21:25:16.723 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 27, 2026-02-17T21:25:16.723 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 35, 2026-02-17T21:25:16.723 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 40, 2026-02-17T21:25:16.723 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 29, 2026-02-17T21:25:16.723 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:25:16.723 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:25:16.723 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:16.723 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.723 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:16.723 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6802", 2026-02-17T21:25:16.723 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2872564052 2026-02-17T21:25:16.723 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.724 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.724 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:16.724 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6803", 2026-02-17T21:25:16.724 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2872564052 2026-02-17T21:25:16.724 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.724 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.724 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.724 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:25:16.724 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:16.724 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.724 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:16.724 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6804", 2026-02-17T21:25:16.724 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2872564052 2026-02-17T21:25:16.725 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.725 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.725 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:16.725 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6805", 2026-02-17T21:25:16.725 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2872564052 2026-02-17T21:25:16.725 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.725 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.725 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.725 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:25:16.725 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:16.725 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.725 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:16.725 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6808", 2026-02-17T21:25:16.725 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2872564052 2026-02-17T21:25:16.726 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.726 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.726 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:16.726 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6809", 2026-02-17T21:25:16.726 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2872564052 2026-02-17T21:25:16.726 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.726 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.726 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.726 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:25:16.726 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:16.726 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.726 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:16.727 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6806", 2026-02-17T21:25:16.727 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2872564052 2026-02-17T21:25:16.727 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.727 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.727 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:16.727 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6807", 2026-02-17T21:25:16.727 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 2872564052 2026-02-17T21:25:16.727 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.727 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.727 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.727 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6803/2872564052", 2026-02-17T21:25:16.727 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6805/2872564052", 2026-02-17T21:25:16.727 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6809/2872564052", 2026-02-17T21:25:16.727 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6807/2872564052", 2026-02-17T21:25:16.728 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:25:16.728 INFO:tasks.workunit.client.0.trial031.stdout: "exists", 2026-02-17T21:25:16.728 INFO:tasks.workunit.client.0.trial031.stdout: "up" 2026-02-17T21:25:16.728 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.728 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.728 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.728 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:25:16.728 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "7650e991-0aa1-446e-810d-5162931dd850", 2026-02-17T21:25:16.728 INFO:tasks.workunit.client.0.trial031.stdout: "up": 0, 2026-02-17T21:25:16.728 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:25:16.728 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:25:16.728 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:25:16.728 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 0, 2026-02-17T21:25:16.729 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 0, 2026-02-17T21:25:16.729 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 10, 2026-02-17T21:25:16.729 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 23, 2026-02-17T21:25:16.729 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 30, 2026-02-17T21:25:16.729 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:25:16.729 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:25:16.729 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:16.729 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.729 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:16.729 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6810", 2026-02-17T21:25:16.729 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:16.729 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.729 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.730 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:16.730 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6811", 2026-02-17T21:25:16.730 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:16.730 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.730 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.730 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.730 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:25:16.730 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:16.730 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.730 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:16.730 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6812", 2026-02-17T21:25:16.730 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:16.730 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.731 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.731 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:16.731 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6813", 2026-02-17T21:25:16.731 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:16.731 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.731 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.731 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.731 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:25:16.731 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:16.731 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.731 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:16.731 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6816", 2026-02-17T21:25:16.731 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:16.732 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.732 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.732 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:16.732 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6817", 2026-02-17T21:25:16.732 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:16.732 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.732 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.732 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.732 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:25:16.732 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:16.732 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.732 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:16.732 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6814", 2026-02-17T21:25:16.733 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:16.733 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.733 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.733 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:16.733 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6815", 2026-02-17T21:25:16.733 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:16.733 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.733 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.733 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.733 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6811/3438498595", 2026-02-17T21:25:16.733 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6813/3438498595", 2026-02-17T21:25:16.733 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3438498595", 2026-02-17T21:25:16.733 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3438498595", 2026-02-17T21:25:16.733 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:25:16.734 INFO:tasks.workunit.client.0.trial031.stdout: "exists" 2026-02-17T21:25:16.734 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.734 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.734 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.734 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:25:16.734 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "a63bbb72-c70c-48fa-b386-d5e9b16de7b4", 2026-02-17T21:25:16.734 INFO:tasks.workunit.client.0.trial031.stdout: "up": 1, 2026-02-17T21:25:16.734 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:25:16.734 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:25:16.734 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:25:16.734 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 14, 2026-02-17T21:25:16.734 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 27, 2026-02-17T21:25:16.734 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 37, 2026-02-17T21:25:16.735 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 0, 2026-02-17T21:25:16.735 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 31, 2026-02-17T21:25:16.735 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:25:16.735 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:25:16.735 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:16.735 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.735 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:16.735 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6810", 2026-02-17T21:25:16.735 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3154825036 2026-02-17T21:25:16.735 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.735 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.736 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:16.736 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6811", 2026-02-17T21:25:16.736 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3154825036 2026-02-17T21:25:16.736 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.736 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.736 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.736 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:25:16.736 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:16.736 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.736 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:16.736 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6812", 2026-02-17T21:25:16.736 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3154825036 2026-02-17T21:25:16.736 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.736 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.737 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:16.737 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6813", 2026-02-17T21:25:16.737 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3154825036 2026-02-17T21:25:16.737 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.737 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.737 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.737 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:25:16.737 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:16.737 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.737 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:16.737 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6816", 2026-02-17T21:25:16.737 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3154825036 2026-02-17T21:25:16.737 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.738 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.738 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:16.738 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6817", 2026-02-17T21:25:16.738 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3154825036 2026-02-17T21:25:16.738 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.738 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.738 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.738 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:25:16.738 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:16.738 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.738 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:16.738 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6814", 2026-02-17T21:25:16.739 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3154825036 2026-02-17T21:25:16.739 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.739 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.739 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:16.739 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6815", 2026-02-17T21:25:16.739 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3154825036 2026-02-17T21:25:16.739 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.739 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.739 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.739 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6811/3154825036", 2026-02-17T21:25:16.739 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6813/3154825036", 2026-02-17T21:25:16.739 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3154825036", 2026-02-17T21:25:16.739 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3154825036", 2026-02-17T21:25:16.739 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:25:16.740 INFO:tasks.workunit.client.0.trial031.stdout: "exists", 2026-02-17T21:25:16.740 INFO:tasks.workunit.client.0.trial031.stdout: "up" 2026-02-17T21:25:16.740 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.740 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.740 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.740 INFO:tasks.workunit.client.0.trial031.stdout: "osd_xinfo": [ 2026-02-17T21:25:16.740 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.740 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:25:16.740 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:24:28.749451+0000", 2026-02-17T21:25:16.740 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:25:16.740 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:25:16.740 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:25:16.740 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:25:16.740 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "2026-02-17T21:23:40.972382+0000", 2026-02-17T21:25:16.741 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:25:16.741 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.741 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.741 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:25:16.741 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:24:29.028147+0000", 2026-02-17T21:25:16.741 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:25:16.741 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:25:16.741 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:25:16.741 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:25:16.741 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-17T21:25:16.741 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:25:16.741 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.741 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.741 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:25:16.741 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:24:29.306966+0000", 2026-02-17T21:25:16.742 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:25:16.742 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:25:16.742 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:25:16.742 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:25:16.742 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "2026-02-17T21:23:58.341558+0000", 2026-02-17T21:25:16.742 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:25:16.742 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.742 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.742 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap": [], 2026-02-17T21:25:16.742 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap_items": [], 2026-02-17T21:25:16.742 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap_primaries": [], 2026-02-17T21:25:16.742 INFO:tasks.workunit.client.0.trial031.stdout: "pg_temp": [], 2026-02-17T21:25:16.742 INFO:tasks.workunit.client.0.trial031.stdout: "primary_temp": [], 2026-02-17T21:25:16.742 INFO:tasks.workunit.client.0.trial031.stdout: "blocklist": {}, 2026-02-17T21:25:16.743 INFO:tasks.workunit.client.0.trial031.stdout: "range_blocklist": {}, 2026-02-17T21:25:16.743 INFO:tasks.workunit.client.0.trial031.stdout: "erasure_code_profiles": { 2026-02-17T21:25:16.743 INFO:tasks.workunit.client.0.trial031.stdout: "default": { 2026-02-17T21:25:16.743 INFO:tasks.workunit.client.0.trial031.stdout: "k": "2", 2026-02-17T21:25:16.743 INFO:tasks.workunit.client.0.trial031.stdout: "m": "2", 2026-02-17T21:25:16.743 INFO:tasks.workunit.client.0.trial031.stdout: "plugin": "jerasure", 2026-02-17T21:25:16.743 INFO:tasks.workunit.client.0.trial031.stdout: "technique": "reed_sol_van" 2026-02-17T21:25:16.743 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.743 INFO:tasks.workunit.client.0.trial031.stdout: "myprofile": { 2026-02-17T21:25:16.743 INFO:tasks.workunit.client.0.trial031.stdout: "crush-device-class": "", 2026-02-17T21:25:16.743 INFO:tasks.workunit.client.0.trial031.stdout: "crush-failure-domain": "osd", 2026-02-17T21:25:16.743 INFO:tasks.workunit.client.0.trial031.stdout: "crush-num-failure-domains": "0", 2026-02-17T21:25:16.743 INFO:tasks.workunit.client.0.trial031.stdout: "crush-osds-per-failure-domain": "0", 2026-02-17T21:25:16.743 INFO:tasks.workunit.client.0.trial031.stdout: "crush-root": "default", 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: "jerasure-per-chunk-alignment": "false", 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: "k": "2", 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: "m": "1", 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: "plugin": "jerasure", 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: "technique": "reed_sol_van", 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: "w": "8" 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: "removed_snaps_queue": [], 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: "new_removed_snaps": [], 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: "new_purged_snaps": [], 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: "crush_node_flags": {}, 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: "device_class_flags": {}, 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode": { 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode_enabled": false, 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_bucket_count": 0, 2026-02-17T21:25:16.744 INFO:tasks.workunit.client.0.trial031.stdout: "degraded_stretch_mode": 0, 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: "recovering_stretch_mode": 0, 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode_bucket": 0 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: "osd_metadata": [ 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6804/2872564052,v1:127.0.0.1:6805/2872564052]", 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:25:16.745 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:25:16.746 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:23:34.448743Z", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6802/2872564052,v1:127.0.0.1:6803/2872564052]", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/2872564052,v1:127.0.0.1:6809/2872564052]", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/2872564052,v1:127.0.0.1:6807/2872564052]", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:25:16.747 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/0", 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6812/3438498595,v1:127.0.0.1:6813/3438498595]", 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:25:16.748 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:23:43.733096Z", 2026-02-17T21:25:16.749 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6810/3438498595,v1:127.0.0.1:6811/3438498595]", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3438498595,v1:127.0.0.1:6817/3438498595]", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3438498595,v1:127.0.0.1:6815/3438498595]", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:25:16.750 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/1", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6812/3154825036,v1:127.0.0.1:6813/3154825036]", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:25:16.751 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:23:51.753333Z", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:25:16.752 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6810/3154825036,v1:127.0.0.1:6811/3154825036]", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3154825036,v1:127.0.0.1:6817/3154825036]", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3154825036,v1:127.0.0.1:6815/3154825036]", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/2", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.753 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_clean_epochs": { 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: "min_last_epoch_clean": 0, 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_clean": { 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: "per_pool": [] 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: "osd_epochs": [ 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 35 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 38 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_first_committed": 1, 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_last_committed": 41, 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: "crushmap": { 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: "devices": [ 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:25:16.754 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.0", 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.1", 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.2", 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: "types": [ 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 0, 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd" 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:25:16.755 INFO:tasks.workunit.client.0.trial031.stdout: "name": "host" 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 2, 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: "name": "chassis" 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 3, 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: "name": "rack" 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 4, 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: "name": "row" 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 5, 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: "name": "pdu" 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 6, 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: "name": "pod" 2026-02-17T21:25:16.756 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 7, 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: "name": "room" 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 8, 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: "name": "datacenter" 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 9, 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: "name": "zone" 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 10, 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: "name": "region" 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: "name": "root" 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.757 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "buckets": [ 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "id": -1, 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "name": "default", 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "root", 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "id": -3, 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "id": -2, 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "name": "default~ssd", 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "root", 2026-02-17T21:25:16.758 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: "id": -4, 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: "id": -3, 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: "name": "trial031", 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "host", 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.759 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 1 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 2 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "id": -4, 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "name": "trial031~ssd", 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "host", 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:25:16.760 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 1 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 2 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: "rules": [ 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: "rule_id": 0, 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: "rule_name": "replicated_rule", 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: "type": 1, 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: "steps": [ 2026-02-17T21:25:16.761 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: "op": "take", 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: "item": -1, 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: "item_name": "default" 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: "op": "choose_firstn", 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: "num": 0, 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: "type": "osd" 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: "op": "emit" 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: "rule_id": 1, 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: "rule_name": "test", 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: "type": 3, 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: "steps": [ 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: "op": "set_chooseleaf_tries", 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: "num": 5 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.762 INFO:tasks.workunit.client.0.trial031.stdout: "op": "set_choose_tries", 2026-02-17T21:25:16.763 INFO:tasks.workunit.client.0.trial031.stdout: "num": 100 2026-02-17T21:25:16.763 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.763 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.763 INFO:tasks.workunit.client.0.trial031.stdout: "op": "take", 2026-02-17T21:25:16.763 INFO:tasks.workunit.client.0.trial031.stdout: "item": -1, 2026-02-17T21:25:16.763 INFO:tasks.workunit.client.0.trial031.stdout: "item_name": "default" 2026-02-17T21:25:16.763 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.763 INFO:tasks.workunit.client.0.trial031.stderr:report 828667832 2026-02-17T21:25:16.763 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.763 INFO:tasks.workunit.client.0.trial031.stdout: "op": "choose_indep", 2026-02-17T21:25:16.763 INFO:tasks.workunit.client.0.trial031.stdout: "num": 0, 2026-02-17T21:25:16.763 INFO:tasks.workunit.client.0.trial031.stdout: "type": "osd" 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "op": "emit" 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "tunables": { 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "choose_local_tries": 0, 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "choose_local_fallback_tries": 0, 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "choose_total_tries": 50, 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_descend_once": 1, 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_vary_r": 1, 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_stable": 1, 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "msr_descents": 100, 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "msr_collision_tries": 100, 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "straw_calc_version": 1, 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "allowed_bucket_algs": 54, 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "profile": "jewel", 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "optimal_tunables": 1, 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "legacy_tunables": 0, 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "minimum_required_version": "jewel", 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables": 1, 2026-02-17T21:25:16.764 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables2": 1, 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "has_v2_rules": 1, 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables3": 1, 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "has_v3_rules": 0, 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "has_v4_buckets": 1, 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables5": 1, 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "has_v5_rules": 0, 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "has_msr_rules": 0 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "choose_args": {} 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "fsmap": { 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 1, 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "btime": "2026-02-17T21:23:32:009201+0000", 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "default_fscid": -1, 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "compat": { 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "compat": {}, 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "ro_compat": {}, 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "incompat": { 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "feature_1": "base v0.20", 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "feature_2": "client writeable ranges", 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "feature_3": "default file layouts on dirs", 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "feature_4": "dir inode in separate object", 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "feature_5": "mds uses versioned encoding", 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-17T21:25:16.765 INFO:tasks.workunit.client.0.trial031.stdout: "feature_8": "no anchor table", 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "feature_9": "file layout v2", 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "feature_10": "snaprealm v2", 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "feature_11": "minor log segments", 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "feature_12": "quiesce subvolumes" 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "feature_flags": { 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "enable_multiple": true, 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "ever_enabled_multiple": true 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "standbys": [], 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "filesystems": [] 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "mdsmap_first_committed": 1, 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "mdsmap_last_committed": 1, 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "auth": { 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "first_committed": 1, 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "last_committed": 5, 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "num_secrets": 3 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg": 1, 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_active": 1, 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_unknown": 0, 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "num_osd": 3, 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "pool_sum": { 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "stat_sum": { 2026-02-17T21:25:16.766 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes": 71567, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects": 101, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_clones": 1, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_copies": 303, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_degraded": 101, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_misplaced": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_unfound": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_dirty": 101, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_whiteouts": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_read": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_read_kb": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_write": 101, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_write_kb": 101, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_scrub_errors": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_deep_scrub_errors": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_recovered": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_recovered": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_keys_recovered": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_omap": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_kb": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_kb": 0, 2026-02-17T21:25:16.767 INFO:tasks.workunit.client.0.trial031.stdout: "num_promote": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_high": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_low": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_some": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_full": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_pinned": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "num_legacy_snapsets": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "num_large_omap_objects": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_manifest": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_bytes": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_keys": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_repaired": 0 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "store_stats": { 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "total": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "available": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 0, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 0 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "log_size": 18, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "ondisk_log_size": 18, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "up": 3, 2026-02-17T21:25:16.768 INFO:tasks.workunit.client.0.trial031.stdout: "acting": 3, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "num_store_stats": 0 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "osd_sum": { 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 0, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 0, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "num_pgs": 3, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "num_osds": 3, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_osds": 3, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "kb": 314572800, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used": 83896, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_data": 2116, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_omap": 10, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_meta": 81717, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "kb_avail": 314488904, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "statfs": { 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "total": 322122547200, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "available": 322036637696, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 2166784, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 1537157, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 10389, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 83679083 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "hb_peers": [], 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "snap_trim_queue_len": 0, 2026-02-17T21:25:16.769 INFO:tasks.workunit.client.0.trial031.stdout: "num_snap_trimming": 0, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "num_shards_repaired": 0, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "op_queue_age_hist": { 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "histogram": [], 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "upper_bound": 1 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "perf_stat": { 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ms": 0, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ms": 0, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ns": 0, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ns": 0 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "alerts": [], 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "network_ping_times": [] 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "osd_sum_by_class": { 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "ssd": { 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 0, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 0, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "num_pgs": 3, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "num_osds": 3, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_osds": 3, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "kb": 314572800, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used": 83896, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_data": 2116, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_omap": 10, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_meta": 81717, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "kb_avail": 314488904, 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "statfs": { 2026-02-17T21:25:16.770 INFO:tasks.workunit.client.0.trial031.stdout: "total": 322122547200, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "available": 322036637696, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 2166784, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 1537157, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 10389, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 83679083 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "hb_peers": [], 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "snap_trim_queue_len": 0, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "num_snap_trimming": 0, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "num_shards_repaired": 0, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "op_queue_age_hist": { 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "histogram": [], 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "upper_bound": 1 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "perf_stat": { 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ms": 0, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ms": 0, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ns": 0, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ns": 0 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "alerts": [], 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "network_ping_times": [] 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "pool_stats": [ 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.771 INFO:tasks.workunit.client.0.trial031.stdout: "poolid": 1, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg": 1, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "stat_sum": { 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes": 71567, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects": 101, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_clones": 1, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_copies": 303, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_degraded": 101, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_misplaced": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_unfound": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_dirty": 101, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_whiteouts": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_read": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_read_kb": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_write": 101, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_write_kb": 101, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_scrub_errors": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_deep_scrub_errors": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_recovered": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_recovered": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_keys_recovered": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_omap": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_kb": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_kb": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_promote": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_high": 0, 2026-02-17T21:25:16.772 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_low": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_some": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_full": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_pinned": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "num_legacy_snapsets": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "num_large_omap_objects": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_manifest": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_bytes": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_keys": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_repaired": 0 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "store_stats": { 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "total": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "available": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 1241088, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 1241088, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 0, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 0 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "log_size": 18, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "ondisk_log_size": 18, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "up": 3, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "acting": 3, 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "num_store_stats": 3 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: "osd_stats": [ 2026-02-17T21:25:16.773 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 150323855371 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 42949672976 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 158913789962 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_by_state": [ 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "state": "active+undersized+degraded", 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "num": 1 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_by_osd": [ 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2147483647, 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 1, 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 1, 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:25:16.774 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 1, 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 1, 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 0, 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snaps": [ 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "pool": 1, 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snaps": {} 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "servicemap": { 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 4, 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:24:50.456146+0000", 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "services": { 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "osd": { 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "daemons": { 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "summary": "", 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "0": { 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "start_epoch": 0, 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "start_stamp": "0.000000", 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "gid": 0, 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "metadata": {}, 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "task_status": {} 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "2": { 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "start_epoch": 0, 2026-02-17T21:25:16.775 INFO:tasks.workunit.client.0.trial031.stdout: "start_stamp": "0.000000", 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: "gid": 0, 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: "metadata": {}, 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: "task_status": {} 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: "mgrstat_first_committed": 1, 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: "mgrstat_last_committed": 50, 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: "logm_first_committed": 1, 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: "logm_last_committed": 248, 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: "paxos": { 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: "first_committed": 1, 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: "last_committed": 349, 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: "last_pn": 0, 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: "accepted_pn": 0 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:16.776 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:25:16.777 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-17T21:25:16.777 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:330: TEST_divergent_ec: for i in $non_divergent 2026-02-17T21:25:16.777 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:332: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.0 2026-02-17T21:25:16.777 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:25:16.777 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:25:16.777 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:25:16.777 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:25:16.777 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:25:16.820 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:25:16.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:333: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 0 --op log --pgid 1.0 2026-02-17T21:25:16.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-17T21:25:16.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T21:25:16.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T21:25:16.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T21:25:16.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/0 2026-02-17T21:25:16.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/0 --op log --pgid 1.0 2026-02-17T21:25:17.479 INFO:tasks.workunit.client.0.trial031.stderr:PG '1.0' not found 2026-02-17T21:25:18.015 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nodown: return 1 2026-02-17T21:25:18.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:334: TEST_divergent_ec: activate_osd td/divergent-priors 0 2026-02-17T21:25:18.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:25:18.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:25:18.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T21:25:18.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:25:18.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/0 2026-02-17T21:25:18.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:25:18.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:25:18.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:25:18.017 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:25:18.017 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-17T21:25:18.017 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-17T21:25:18.017 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:25:18.017 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:25:18.017 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:25:18.017 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:25:18.017 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:25:18.017 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:25:18.017 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:25:18.017 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:25:18.018 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:25:18.018 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:25:18.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:25:18.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:25:18.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:25:18.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:25:18.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:25:18.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:25:18.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:25:18.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:25:18.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:25:18.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:25:18.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/0 2026-02-17T21:25:18.019 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:25:18.020 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T21:25:18.020 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:25:18.020 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/0/whoami 2026-02-17T21:25:18.020 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T21:25:18.020 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:25:18.021 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:25:18.021 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:25:18.042 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:25:18.037+0000 7f4b69b458c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:25:18.043 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:25:18.041+0000 7f4b69b458c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:25:18.044 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:25:18.041+0000 7f4b69b458c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:25:18.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T21:25:18.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:25:18.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:25:18.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:25:18.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:25:18.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:25:18.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:25:18.220 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:25:18.221 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:25:18.221 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:25:18.426 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 35 up_thru 40 down_at 29 last_clean_interval [5,27) [v2:127.0.0.1:6802/2872564052,v1:127.0.0.1:6803/2872564052] [v2:127.0.0.1:6804/2872564052,v1:127.0.0.1:6805/2872564052] exists,up 27052348-33c5-4b28-9883-30bfd1415320 2026-02-17T21:25:18.428 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:25:18.428 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:25:18.428 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:25:18.428 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:330: TEST_divergent_ec: for i in $non_divergent 2026-02-17T21:25:18.428 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:332: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.2 2026-02-17T21:25:18.428 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:25:18.428 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:25:18.428 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:25:18.429 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:25:18.429 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:25:18.535 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:25:18.535 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:333: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 2 --op log --pgid 1.0 2026-02-17T21:25:18.535 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-17T21:25:18.535 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T21:25:18.535 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T21:25:18.536 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T21:25:18.536 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/2 2026-02-17T21:25:18.536 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/2 --op log --pgid 1.0 2026-02-17T21:25:19.191 INFO:tasks.workunit.client.0.trial031.stderr:PG '1.0' not found 2026-02-17T21:25:19.325 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:25:19.321+0000 7f4b69b458c0 -1 Falling back to public interface 2026-02-17T21:25:19.728 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nodown: return 1 2026-02-17T21:25:19.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:334: TEST_divergent_ec: activate_osd td/divergent-priors 2 2026-02-17T21:25:19.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:25:19.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:25:19.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T21:25:19.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:25:19.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/2 2026-02-17T21:25:19.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:25:19.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:25:19.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:25:19.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:25:19.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-17T21:25:19.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-17T21:25:19.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:25:19.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:25:19.729 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:25:19.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/2 2026-02-17T21:25:19.732 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:25:19.732 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T21:25:19.732 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:25:19.732 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/2/whoami 2026-02-17T21:25:19.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T21:25:19.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:25:19.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:25:19.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:25:19.753 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:25:19.749+0000 7fa9767e78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:25:19.754 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:25:19.749+0000 7fa9767e78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:25:19.755 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:25:19.753+0000 7fa9767e78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:25:19.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T21:25:19.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:25:19.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:25:19.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:25:19.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:25:19.942 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:25:19.942 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:25:19.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:25:19.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:25:19.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:25:20.142 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 37 up_thru 0 down_at 31 last_clean_interval [14,27) [v2:127.0.0.1:6810/3154825036,v1:127.0.0.1:6811/3154825036] [v2:127.0.0.1:6812/3154825036,v1:127.0.0.1:6813/3154825036] exists,up a63bbb72-c70c-48fa-b386-d5e9b16de7b4 2026-02-17T21:25:20.143 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:25:20.143 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:25:20.143 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:25:20.143 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:336: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 1 --op log --pgid 1.0 2026-02-17T21:25:20.143 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-17T21:25:20.143 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T21:25:20.144 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T21:25:20.144 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T21:25:20.144 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-17T21:25:20.144 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op log --pgid 1.0 2026-02-17T21:25:20.802 INFO:tasks.workunit.client.0.trial031.stderr:PG '1.0' not found 2026-02-17T21:25:20.997 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:25:20.993+0000 7fa9767e78c0 -1 Falling back to public interface 2026-02-17T21:25:21.341 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nodown: return 1 2026-02-17T21:25:21.341 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:338: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-17T21:25:21.341 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:338: TEST_divergent_ec: wait_for_clean 2026-02-17T21:25:21.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:25:21.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:25:21.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:25:21.342 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-17T21:25:21.343 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:25:21.343 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:25:21.343 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:25:21.343 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:25:21.343 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:25:21.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-17T21:25:21.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:25:21.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:25:21.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:25:21.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:25:21.404 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:25:21.610 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:25:21.611 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:25:21.611 INFO:tasks.workunit.client.0.trial031.stderr:2' 2026-02-17T21:25:21.611 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:25:21.611 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:25:21.611 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:25:22.357 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:25:22.353+0000 7f4b69b458c0 -1 osd.0 41 log_to_monitors true 2026-02-17T21:25:23.984 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:25:23.981+0000 7fa9767e78c0 -1 osd.2 41 log_to_monitors true 2026-02-17T21:25:26.694 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561027 2026-02-17T21:25:26.694 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561027 2026-02-17T21:25:26.694 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561027' 2026-02-17T21:25:26.694 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:25:26.694 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:25:26.753 INFO:tasks.workunit.client.0.trial031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-17T21:25:26.757 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-17T21:25:26.757 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-17T21:25:26.757 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-17T21:25:26.757 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:25:26.757 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:25:26.835 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=206158430211 2026-02-17T21:25:26.835 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 206158430211 2026-02-17T21:25:26.835 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561027 2-206158430211' 2026-02-17T21:25:26.836 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:25:26.836 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-188978561027 2026-02-17T21:25:26.836 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:25:26.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:25:26.838 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-188978561027 2026-02-17T21:25:26.838 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:25:26.839 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561027 2026-02-17T21:25:26.840 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 188978561027 2026-02-17T21:25:26.840 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 188978561027' 2026-02-17T21:25:26.840 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:25:27.042 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 188978561027 2026-02-17T21:25:27.042 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:25:28.043 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:25:28.043 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:25:28.246 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 188978561027 2026-02-17T21:25:28.247 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:25:29.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T21:25:29.248 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:25:29.459 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561028 -lt 188978561027 2026-02-17T21:25:29.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:25:29.460 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-206158430211 2026-02-17T21:25:29.460 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:25:29.462 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:25:29.462 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-206158430211 2026-02-17T21:25:29.462 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:25:29.464 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=206158430211 2026-02-17T21:25:29.464 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 206158430211 2026-02-17T21:25:29.464 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 206158430211' 2026-02-17T21:25:29.465 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:25:29.665 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 206158430211 -lt 206158430211 2026-02-17T21:25:29.666 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:25:29.666 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:25:29.666 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:25:29.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T21:25:29.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:25:29.926 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:25:29.926 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:25:29.927 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:25:29.927 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:25:29.927 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:25:29.927 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:25:30.130 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:25:30.131 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:25:30.131 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:25:30.131 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:25:30.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:25:30.402 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T21:25:30.402 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T21:25:30.402 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T21:25:30.402 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:25:30.402 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T21:25:30.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:25:30.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:25:30.503 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:25:30.503 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:25:30.503 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:25:30.503 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:25:30.504 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:25:30.504 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:25:30.705 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:25:30.706 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:25:30.706 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:25:30.706 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:25:30.966 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:25:30.966 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:25:30.966 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:25:30.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:25:30.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:25:30.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:25:30.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:25:30.967 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:25:30.967 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:25:31.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:25:31.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:25:31.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-17T21:25:31.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:25:31.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T21:25:31.439 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:25:31.439 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:25:31.440 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:25:31.440 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:25:31.440 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:25:31.440 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:25:31.440 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:25:31.440 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:25:31.634 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:25:31.635 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:25:31.635 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:25:31.635 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:25:31.891 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:25:31.892 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:25:31.892 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:25:31.892 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:25:31.892 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:25:31.892 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:25:31.892 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:25:31.892 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:25:31.893 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:25:32.154 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:25:32.155 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:25:32.155 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-17T21:25:32.155 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:25:32.155 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T21:25:32.555 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:25:32.555 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:25:32.556 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:25:32.556 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:25:32.556 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:25:32.557 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:25:32.557 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:25:32.557 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:25:32.756 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:25:32.757 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:25:32.757 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:25:32.757 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:25:33.013 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:25:33.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:25:33.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:25:33.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:25:33.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:25:33.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:25:33.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:25:33.015 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:25:33.015 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:25:33.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:25:33.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:25:33.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-17T21:25:33.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:25:33.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T21:25:34.073 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:25:34.073 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:25:34.074 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:25:34.074 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:25:34.074 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:25:34.075 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:25:34.075 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:25:34.075 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:25:34.270 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:25:34.271 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:25:34.271 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:25:34.271 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:25:34.537 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:25:34.537 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:25:34.537 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:25:34.538 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:25:34.538 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:25:34.538 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:25:34.538 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:25:34.538 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:25:34.539 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:25:34.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:25:34.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:25:34.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-17T21:25:34.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:25:34.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-17T21:25:36.395 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:25:36.395 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:25:36.396 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:25:36.396 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:25:36.397 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:25:36.397 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:25:36.397 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:25:36.397 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:25:36.594 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:25:36.595 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:25:36.595 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:25:36.595 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:25:36.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:25:36.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:25:36.857 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:25:36.857 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:25:36.857 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:25:36.857 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:25:36.857 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:25:36.857 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:25:36.858 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:25:37.112 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:25:37.113 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:25:37.113 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-17T21:25:37.113 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:25:37.113 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-17T21:25:40.313 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:25:40.314 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:25:40.315 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:25:40.315 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:25:40.315 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:25:40.315 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:25:40.315 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:25:40.315 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:25:40.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:25:40.513 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:25:40.513 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:25:40.513 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:25:40.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:25:40.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:25:40.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:25:40.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:25:40.775 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:25:40.775 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:25:40.775 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:25:40.775 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:25:40.775 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:25:41.029 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:25:41.029 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:25:41.029 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-17T21:25:41.029 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:25:41.029 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-17T21:25:47.429 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:25:47.430 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:25:47.431 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:25:47.431 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:25:47.431 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:25:47.431 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:25:47.431 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:25:47.431 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:25:47.628 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:25:47.628 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:25:47.629 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:25:47.629 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:25:47.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:25:47.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:25:47.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:25:47.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:25:47.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:25:47.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:25:47.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:25:47.885 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:25:47.885 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:25:48.159 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:25:48.159 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:25:48.159 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-17T21:25:48.159 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T21:25:48.159 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-17T21:25:55.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T21:25:55.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:25:55.461 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:25:55.461 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:25:55.462 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:25:55.462 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:25:55.463 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:25:55.463 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:25:55.664 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T21:25:55.665 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:25:55.666 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:25:55.666 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:25:55.935 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T21:25:55.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T21:25:55.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T21:25:55.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T21:25:55.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T21:25:55.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T21:25:55.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T21:25:55.936 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T21:25:55.936 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T21:25:56.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T21:25:56.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T21:25:56.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-17T21:25:56.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_fingerprint": "a569169b-2a17-4b91-b9bb-24e9547abd0e", 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "version": "19.2.3-679-gd855f53b", 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "commit": "d855f53b89fdcec760fd9232a5fb55ed4fb111a1", 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "timestamp": "2026-02-17T21:25:56.460686+0000", 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "tag": "", 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "health": { 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "status": "HEALTH_WARN", 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "checks": { 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "OSDMAP_FLAGS": { 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "count": 17 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [], 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "OSD_DOWN": { 2026-02-17T21:25:56.460 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "message": "1 osds down", 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "message": "osd.1 (root=default,host=trial031) is down" 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "PG_DEGRADED": { 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "message": "Degraded data redundancy: 101/303 objects degraded (33.333%), 1 pg degraded", 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [NONE,0,2]" 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "severity": "HEALTH_WARN", 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "summary": { 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "count": 1 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "detail": [ 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.461 INFO:tasks.workunit.client.0.trial031.stdout: "message": "application not enabled on pool 'test'" 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "muted": false 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "mutes": [] 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "monmap_first_committed": 1, 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "monmap_last_committed": 1, 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "monmap": { 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 1, 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "fsid": "5ebac6e7-5321-4893-ad68-c06127ed455b", 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:23:31.926448Z", 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "created": "2026-02-17T21:23:31.926448Z", 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "min_mon_release": 19, 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "min_mon_release_name": "squid", 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "election_strategy": 1, 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "disallowed_leaders": "", 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode": false, 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "tiebreaker_mon": "", 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "removed_ranks": "", 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "features": { 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "persistent": [ 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "kraken", 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "luminous", 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "mimic", 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap-prune", 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "nautilus", 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "octopus", 2026-02-17T21:25:56.462 INFO:tasks.workunit.client.0.trial031.stdout: "pacific", 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "elector-pinging", 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "quincy", 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "reef", 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "squid" 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "optional": [] 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "mons": [ 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "rank": 0, 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "name": "a", 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:7115", 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 0 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:7115/0", 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "priority": 0, 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 0, 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "crush_location": "{}" 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "quorum": [ 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: 0 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap": { 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 49, 2026-02-17T21:25:56.463 INFO:tasks.workunit.client.0.trial031.stdout: "fsid": "5ebac6e7-5321-4893-ad68-c06127ed455b", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "created": "2026-02-17T21:23:32.009422+0000", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:25:24.376358+0000", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "last_up_change": "2026-02-17T21:25:24.273669+0000", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "last_in_change": "2026-02-17T21:24:41.116674+0000", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "flags_num": 5800384, 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "flags_set": [ 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "nodown", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "noin", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "noout", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "pglog_hardlimit", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snapdirs", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "recovery_deletes", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "sortbitwise" 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "crush_version": 12, 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "full_ratio": 0.99000000953674316, 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_snapshot": "", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "pool_max": 1, 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "max_osd": 3, 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "require_min_compat_client": "luminous", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "min_compat_client": "jewel", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "require_osd_release": "squid", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "allow_crimson": false, 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "pools": [ 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "pool": 1, 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "pool_name": "test", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "create_time": "2026-02-17T21:24:00.416440+0000", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "flags": 16389, 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "flags_names": "hashpspool,ec_overwrites,pool_snaps", 2026-02-17T21:25:56.464 INFO:tasks.workunit.client.0.trial031.stdout: "type": 3, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "size": 3, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "min_size": 2, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "crush_rule": 1, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_count": 0, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_target": 0, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_barrier": 0, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "is_stretch_pool": false, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "object_hash": 2, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "pg_autoscale_mode": "off", 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num": 1, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "pg_placement_num": 1, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "pg_placement_num_target": 1, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num_target": 1, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "pg_num_pending": 1, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "last_pg_merge_meta": { 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "source_pgid": "0.0", 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "ready_epoch": 0, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_started": 0, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_clean": 0, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "source_version": "0'0", 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "target_version": "0'0" 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "last_change": "28", 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend": "0", 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "last_force_op_resend_preluminous": "0", 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "auid": 0, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "snap_mode": "pool", 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "snap_seq": 1, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "snap_epoch": 28, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "pool_snaps": [ 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "snapid": 1, 2026-02-17T21:25:56.465 INFO:tasks.workunit.client.0.trial031.stdout: "stamp": "2026-02-17T21:24:18.130543+0000", 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "name": "snap1" 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "removed_snaps": "[]", 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "quota_max_bytes": 0, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "quota_max_objects": 0, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "tiers": [], 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "tier_of": -1, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "read_tier": -1, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "write_tier": -1, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "cache_mode": "none", 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "target_max_bytes": 0, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "target_max_objects": 0, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "cache_min_flush_age": 0, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "cache_min_evict_age": 0, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "erasure_code_profile": "myprofile", 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_params": { 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "type": "none" 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_period": 0, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_count": 0, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "use_gmt_hitset": true, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "min_read_recency_for_promote": 0, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "min_write_recency_for_promote": 0, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_grade_decay_rate": 0, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "hit_set_search_last_n": 0, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "grade_table": [], 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "stripe_width": 8192, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "expected_num_objects": 0, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "fast_read": false, 2026-02-17T21:25:56.466 INFO:tasks.workunit.client.0.trial031.stdout: "options": {}, 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "application_metadata": {} 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "osds": [ 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "27052348-33c5-4b28-9883-30bfd1415320", 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "up": 1, 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 35, 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 41, 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 44, 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 48, 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 43, 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6802", 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 4003928214 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6803", 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 4003928214 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:56.467 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6804", 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 4003928214 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6805", 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 4003928214 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6808", 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 4003928214 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6809", 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 4003928214 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.468 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6806", 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 4003928214 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6807", 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 4003928214 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6803/4003928214", 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6805/4003928214", 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6809/4003928214", 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6807/4003928214", 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: "exists", 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: "up" 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.469 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "7650e991-0aa1-446e-810d-5162931dd850", 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "up": 0, 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 0, 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 0, 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 10, 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 23, 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 30, 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:56.470 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6810", 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6811", 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6812", 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:56.471 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6813", 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6816", 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6817", 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.472 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6814", 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6815", 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3438498595 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6811/3438498595", 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6813/3438498595", 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3438498595", 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3438498595", 2026-02-17T21:25:56.473 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: "exists" 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: "uuid": "a63bbb72-c70c-48fa-b386-d5e9b16de7b4", 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: "up": 1, 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: "in": 1, 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 1, 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: "primary_affinity": 1, 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_begin": 37, 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: "last_clean_end": 41, 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 48, 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: "up_thru": 0, 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: "down_at": 47, 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: "lost_at": 0, 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: "public_addrs": { 2026-02-17T21:25:56.474 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6810", 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3527174734 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6811", 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3527174734 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addrs": { 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6812", 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3527174734 2026-02-17T21:25:56.475 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6813", 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3527174734 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addrs": { 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6816", 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3527174734 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6817", 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3527174734 2026-02-17T21:25:56.476 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addrs": { 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: "addrvec": [ 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v2", 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6814", 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3527174734 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: "type": "v1", 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "127.0.0.1:6815", 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: "nonce": 3527174734 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: "public_addr": "127.0.0.1:6811/3527174734", 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: "cluster_addr": "127.0.0.1:6813/3527174734", 2026-02-17T21:25:56.477 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3527174734", 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3527174734", 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: "state": [ 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: "exists", 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: "up" 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: "osd_xinfo": [ 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:25:22.513976+0000", 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "2026-02-17T21:23:40.972382+0000", 2026-02-17T21:25:56.478 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:24:29.028147+0000", 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: "down_stamp": "2026-02-17T21:25:24.173098+0000", 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_probability": 0, 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: "laggy_interval": 0, 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: "features": 4540701547738038271, 2026-02-17T21:25:56.479 INFO:tasks.workunit.client.0.trial031.stdout: "old_weight": 0, 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: "last_purged_snaps_scrub": "2026-02-17T21:23:58.341558+0000", 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: "dead_epoch": 0 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap": [], 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap_items": [], 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: "pg_upmap_primaries": [], 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: "pg_temp": [], 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: "primary_temp": [], 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: "blocklist": {}, 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: "range_blocklist": {}, 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: "erasure_code_profiles": { 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: "default": { 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: "k": "2", 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: "m": "2", 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: "plugin": "jerasure", 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: "technique": "reed_sol_van" 2026-02-17T21:25:56.480 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "myprofile": { 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "crush-device-class": "", 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "crush-failure-domain": "osd", 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "crush-num-failure-domains": "0", 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "crush-osds-per-failure-domain": "0", 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "crush-root": "default", 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "jerasure-per-chunk-alignment": "false", 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "k": "2", 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "m": "1", 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "plugin": "jerasure", 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "technique": "reed_sol_van", 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "w": "8" 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "removed_snaps_queue": [], 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "new_removed_snaps": [], 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "new_purged_snaps": [], 2026-02-17T21:25:56.481 INFO:tasks.workunit.client.0.trial031.stdout: "crush_node_flags": {}, 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "device_class_flags": {}, 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode": { 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode_enabled": false, 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_bucket_count": 0, 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "degraded_stretch_mode": 0, 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "recovering_stretch_mode": 0, 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "stretch_mode_bucket": 0 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "osd_metadata": [ 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6804/4003928214,v1:127.0.0.1:6805/4003928214]", 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:25:56.482 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:23:34.448743Z", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:25:56.483 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6802/4003928214,v1:127.0.0.1:6803/4003928214]", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/4003928214,v1:127.0.0.1:6809/4003928214]", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/4003928214,v1:127.0.0.1:6807/4003928214]", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/0", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.484 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6812/3438498595,v1:127.0.0.1:6813/3438498595]", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:25:56.485 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:23:43.733096Z", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6810/3438498595,v1:127.0.0.1:6811/3438498595]", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3438498595,v1:127.0.0.1:6817/3438498595]", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3438498595,v1:127.0.0.1:6815/3438498595]", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:25:56.486 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/1", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "arch": "x86_64", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "back_addr": "[v2:127.0.0.1:6812/3527174734,v1:127.0.0.1:6813/3527174734]", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "back_iface": "", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs": "1", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_db": "0", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_dedicated_wal": "0", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "bluefs_single_shared_device": "1", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocation_from_file": "1", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_allocator": "hybrid", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_access_mode": "file", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_block_size": "4096", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-17T21:25:56.487 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_rotational": "0", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_size": "107374182400", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_support_discard": "1", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_bdev_type": "ssd", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "bluestore_min_alloc_size": "4096", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_release": "squid", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_short": "19.2.3-679-gd855f53b", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "ceph_version_when_created": "ceph version 19.2.3-679-gd855f53b (d855f53b89fdcec760fd9232a5fb55ed4fb111a1) squid (stable)", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "created_at": "2026-02-17T21:23:51.753333Z", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "default_device_class": "ssd", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6BC1", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "devices": "nvme0n1", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "distro": "ubuntu", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "distro_version": "22.04", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "front_addr": "[v2:127.0.0.1:6810/3527174734,v1:127.0.0.1:6811/3527174734]", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "front_iface": "", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3527174734,v1:127.0.0.1:6817/3527174734]", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3527174734,v1:127.0.0.1:6815/3527174734]", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "hostname": "trial031", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "journal_rotational": "0", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "mem_swap_kb": "8388604", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "mem_total_kb": "130961704", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_node": "0", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "objectstore_numa_nodes": "0", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "os": "Linux", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "osd_data": "td/divergent-priors/2", 2026-02-17T21:25:56.488 INFO:tasks.workunit.client.0.trial031.stdout: "osd_objectstore": "bluestore", 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "osdspec_affinity": "", 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "rotational": "0" 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_clean_epochs": { 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "min_last_epoch_clean": 24, 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "last_epoch_clean": { 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "per_pool": [ 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "poolid": 1, 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "floor": 24 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "osd_epochs": [ 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 45 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 49 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_first_committed": 1, 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "osdmap_last_committed": 49, 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "crushmap": { 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "devices": [ 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.0", 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.489 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.1", 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd.2", 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "class": "ssd" 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "types": [ 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 0, 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "name": "osd" 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "name": "host" 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 2, 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "name": "chassis" 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 3, 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "name": "rack" 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 4, 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "name": "row" 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 5, 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: "name": "pdu" 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.490 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 6, 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "name": "pod" 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 7, 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "name": "room" 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 8, 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "name": "datacenter" 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 9, 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "name": "zone" 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 10, 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "name": "region" 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "name": "root" 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "buckets": [ 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "id": -1, 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "name": "default", 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "root", 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.491 INFO:tasks.workunit.client.0.trial031.stdout: "id": -3, 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "id": -2, 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "name": "default~ssd", 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 11, 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "root", 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "id": -4, 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "id": -3, 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "name": "trial031", 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "host", 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.492 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 1 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 2 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "id": -4, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "name": "trial031~ssd", 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "type_id": 1, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "type_name": "host", 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 19206, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "alg": "straw2", 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "hash": "rjenkins1", 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "items": [ 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "id": 0, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 0 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "id": 1, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 1 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "id": 2, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "weight": 6402, 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: "pos": 2 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.493 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "rules": [ 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "rule_id": 0, 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "rule_name": "replicated_rule", 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "type": 1, 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "steps": [ 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "op": "take", 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "item": -1, 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "item_name": "default" 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "op": "choose_firstn", 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "num": 0, 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "type": "osd" 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "op": "emit" 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "rule_id": 1, 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "rule_name": "test", 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "type": 3, 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "steps": [ 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "op": "set_chooseleaf_tries", 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "num": 5 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "op": "set_choose_tries", 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "num": 100 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.494 INFO:tasks.workunit.client.0.trial031.stdout: "op": "take", 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stderr:report 2539124762 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: "item": -1, 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: "item_name": "default" 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: "op": "choose_indep", 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: "num": 0, 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: "type": "osd" 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: "op": "emit" 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: ] 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: "tunables": { 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: "choose_local_tries": 0, 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: "choose_local_fallback_tries": 0, 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: "choose_total_tries": 50, 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_descend_once": 1, 2026-02-17T21:25:56.495 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_vary_r": 1, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "chooseleaf_stable": 1, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "msr_descents": 100, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "msr_collision_tries": 100, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "straw_calc_version": 1, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "allowed_bucket_algs": 54, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "profile": "jewel", 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "optimal_tunables": 1, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "legacy_tunables": 0, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "minimum_required_version": "jewel", 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables": 1, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables2": 1, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "has_v2_rules": 1, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables3": 1, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "has_v3_rules": 0, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "has_v4_buckets": 1, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "require_feature_tunables5": 1, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "has_v5_rules": 0, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "has_msr_rules": 0 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "choose_args": {} 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "fsmap": { 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 1, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "btime": "2026-02-17T21:23:32:009201+0000", 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "default_fscid": -1, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "compat": { 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "compat": {}, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "ro_compat": {}, 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "incompat": { 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "feature_1": "base v0.20", 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "feature_2": "client writeable ranges", 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "feature_3": "default file layouts on dirs", 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "feature_4": "dir inode in separate object", 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "feature_5": "mds uses versioned encoding", 2026-02-17T21:25:56.496 INFO:tasks.workunit.client.0.trial031.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "feature_8": "no anchor table", 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "feature_9": "file layout v2", 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "feature_10": "snaprealm v2", 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "feature_11": "minor log segments", 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "feature_12": "quiesce subvolumes" 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "feature_flags": { 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "enable_multiple": true, 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "ever_enabled_multiple": true 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "standbys": [], 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "filesystems": [] 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "mdsmap_first_committed": 1, 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "mdsmap_last_committed": 1, 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "auth": { 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "first_committed": 1, 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "last_committed": 5, 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "num_secrets": 3 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg": 1, 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_active": 1, 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_unknown": 0, 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "num_osd": 3, 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "pool_sum": { 2026-02-17T21:25:56.497 INFO:tasks.workunit.client.0.trial031.stdout: "stat_sum": { 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes": 71567, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects": 101, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_clones": 1, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_copies": 303, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing": 0, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_degraded": 101, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_misplaced": 0, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_unfound": 0, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_dirty": 101, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_whiteouts": 0, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_read": 0, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_read_kb": 0, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_write": 101, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_write_kb": 101, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_scrub_errors": 0, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_deep_scrub_errors": 0, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_recovered": 0, 2026-02-17T21:25:56.498 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_recovered": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_keys_recovered": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_omap": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_kb": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_kb": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_promote": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_high": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_low": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_some": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_full": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_pinned": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_legacy_snapsets": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_large_omap_objects": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_manifest": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_bytes": 0, 2026-02-17T21:25:56.499 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_keys": 0, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_repaired": 0 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "store_stats": { 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "total": 0, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "available": 0, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 0, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 0, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 0, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 0 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "log_size": 18, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "ondisk_log_size": 18, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "up": 3, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "acting": 3, 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: "num_store_stats": 0 2026-02-17T21:25:56.500 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "osd_sum": { 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 0, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 0, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "num_pgs": 3, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "num_osds": 3, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_osds": 3, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "kb": 314572800, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used": 85080, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_data": 2244, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_omap": 15, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_meta": 82736, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "kb_avail": 314487720, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "statfs": { 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "total": 322122547200, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "available": 322035425280, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 2297856, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 1592025, 2026-02-17T21:25:56.501 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 16250, 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 84721798 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "hb_peers": [], 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "snap_trim_queue_len": 0, 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "num_snap_trimming": 0, 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "num_shards_repaired": 0, 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "op_queue_age_hist": { 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "histogram": [], 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "upper_bound": 1 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "perf_stat": { 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ms": 0, 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ms": 0, 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ns": 0, 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ns": 0 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "alerts": [], 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "network_ping_times": [] 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.502 INFO:tasks.workunit.client.0.trial031.stdout: "osd_sum_by_class": { 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "ssd": { 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "up_from": 0, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 0, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "num_pgs": 3, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "num_osds": 3, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_osds": 3, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "kb": 314572800, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used": 85080, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_data": 2244, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_omap": 15, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "kb_used_meta": 82736, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "kb_avail": 314487720, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "statfs": { 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "total": 322122547200, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "available": 322035425280, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 2297856, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 1592025, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:25:56.503 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 16250, 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 84721798 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "hb_peers": [], 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "snap_trim_queue_len": 0, 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "num_snap_trimming": 0, 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "num_shards_repaired": 0, 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "op_queue_age_hist": { 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "histogram": [], 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "upper_bound": 1 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "perf_stat": { 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ms": 0, 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ms": 0, 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "commit_latency_ns": 0, 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "apply_latency_ns": 0 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "alerts": [], 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "network_ping_times": [] 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "pool_stats": [ 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.504 INFO:tasks.workunit.client.0.trial031.stdout: "poolid": 1, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg": 1, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "stat_sum": { 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes": 71567, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects": 101, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_clones": 1, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_object_copies": 303, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_missing": 0, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_degraded": 101, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_misplaced": 0, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_unfound": 0, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_dirty": 101, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_whiteouts": 0, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_read": 0, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_read_kb": 0, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_write": 101, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_write_kb": 101, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_scrub_errors": 0, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_deep_scrub_errors": 0, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_recovered": 0, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_recovered": 0, 2026-02-17T21:25:56.505 INFO:tasks.workunit.client.0.trial031.stdout: "num_keys_recovered": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_omap": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_kb": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_kb": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_promote": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_high": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_flush_mode_low": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_some": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_evict_mode_full": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_pinned": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_legacy_snapsets": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_large_omap_objects": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_manifest": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_bytes": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_omap_keys": 0, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "num_objects_repaired": 0 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "store_stats": { 2026-02-17T21:25:56.506 INFO:tasks.workunit.client.0.trial031.stdout: "total": 0, 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "available": 0, 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "internally_reserved": 0, 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "allocated": 1241088, 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "data_stored": 1241088, 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed": 0, 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_allocated": 0, 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "data_compressed_original": 0, 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "omap_allocated": 0, 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "internal_metadata": 0 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "log_size": 18, 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "ondisk_log_size": 18, 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "up": 3, 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "acting": 3, 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "num_store_stats": 3 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "osd_stats": [ 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:25:56.507 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 188978561033 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 42949672976 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: "seq": 206158430217 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_by_state": [ 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: "state": "active+undersized+degraded", 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: "num": 1 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: "num_pg_by_osd": [ 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2147483647, 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 1, 2026-02-17T21:25:56.508 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 2, 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 1, 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 0, 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 1, 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 1, 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: "osd": 1, 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: "num_primary_pg": 0, 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: "num_acting_pg": 0, 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: "num_up_not_acting_pg": 0 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snaps": [ 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: { 2026-02-17T21:25:56.509 INFO:tasks.workunit.client.0.trial031.stdout: "pool": 1, 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "purged_snaps": {} 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: ], 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "servicemap": { 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "epoch": 4, 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "modified": "2026-02-17T21:24:50.456146+0000", 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "services": { 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "osd": { 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "daemons": { 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "summary": "", 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "0": { 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "start_epoch": 0, 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "start_stamp": "0.000000", 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "gid": 0, 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "metadata": {}, 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "task_status": {} 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "2": { 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "start_epoch": 0, 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "start_stamp": "0.000000", 2026-02-17T21:25:56.510 INFO:tasks.workunit.client.0.trial031.stdout: "gid": 0, 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: "metadata": {}, 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: "task_status": {} 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: }, 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: "mgrstat_first_committed": 1, 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: "mgrstat_last_committed": 69, 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: "logm_first_committed": 1, 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: "logm_last_committed": 327, 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: "paxos": { 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: "first_committed": 1, 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: "last_committed": 454, 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: "last_pn": 0, 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: "accepted_pn": 0 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout: } 2026-02-17T21:25:56.511 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:25:56.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-17T21:25:56.512 INFO:tasks.workunit.client.0.trial031.stdout:delay recovery 2026-02-17T21:25:56.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:341: TEST_divergent_ec: echo 'delay recovery' 2026-02-17T21:25:56.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:342: TEST_divergent_ec: ceph pg dump pgs 2026-02-17T21:25:56.668 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:25:56.668 INFO:tasks.workunit.client.0.trial031.stdout:1.0 101 0 101 0 0 71567 0 0 18 84 18 active+undersized+degraded 2026-02-17T21:25:24.385137+0000 41'102 49:140 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-17T21:24:00.509416+0000 0'0 2026-02-17T21:24:00.509416+0000 0 0 no scrub is scheduled 0 0 2026-02-17T21:25:56.669 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:25:56.669 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:25:56.669 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:25:56.677 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:343: TEST_divergent_ec: for i in $non_divergent 2026-02-17T21:25:56.677 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: get_asok_path osd.0 2026-02-17T21:25:56.677 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T21:25:56.677 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T21:25:56.678 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:25:56.678 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:25:56.678 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:25:56.678 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.0.asok 2026-02-17T21:25:56.679 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: CEPH_ARGS= 2026-02-17T21:25:56.679 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.0.asok set_recovery_delay 100000 2026-02-17T21:25:56.734 INFO:tasks.workunit.client.0.trial031.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-17T21:25:56.739 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:343: TEST_divergent_ec: for i in $non_divergent 2026-02-17T21:25:56.740 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: get_asok_path osd.2 2026-02-17T21:25:56.740 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-17T21:25:56.740 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-17T21:25:56.740 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:25:56.740 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:25:56.740 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:25:56.741 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.2.asok 2026-02-17T21:25:56.741 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: CEPH_ARGS= 2026-02-17T21:25:56.741 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.2.asok set_recovery_delay 100000 2026-02-17T21:25:56.793 INFO:tasks.workunit.client.0.trial031.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-17T21:25:56.799 INFO:tasks.workunit.client.0.trial031.stdout:revive divergent 1 2026-02-17T21:25:56.799 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:349: TEST_divergent_ec: echo 'revive divergent 1' 2026-02-17T21:25:56.799 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:350: TEST_divergent_ec: ceph pg dump pgs 2026-02-17T21:25:56.998 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:25:56.999 INFO:tasks.workunit.client.0.trial031.stdout:1.0 101 0 101 0 0 71567 0 0 18 84 18 active+undersized+degraded 2026-02-17T21:25:24.385137+0000 41'102 49:140 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-17T21:24:00.509416+0000 0'0 2026-02-17T21:24:00.509416+0000 0 0 no scrub is scheduled 0 0 2026-02-17T21:25:56.999 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:25:56.999 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:25:56.999 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:25:57.007 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:351: TEST_divergent_ec: ceph osd set noup 2026-02-17T21:25:57.324 INFO:tasks.workunit.client.0.trial031.stderr:noup is set 2026-02-17T21:25:57.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:352: TEST_divergent_ec: activate_osd td/divergent-priors 1 2026-02-17T21:25:57.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:25:57.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:25:57.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T21:25:57.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:25:57.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-17T21:25:57.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:25:57.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:25:57.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:25:57.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:25:57.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-17T21:25:57.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-17T21:25:57.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:25:57.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:25:57.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:25:57.338 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:25:57.338 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:25:57.338 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:25:57.338 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:25:57.338 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:25:57.338 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:25:57.339 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:25:57.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:25:57.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:25:57.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:25:57.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:25:57.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:25:57.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:25:57.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:25:57.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:25:57.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:25:57.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:25:57.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-17T21:25:57.339 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:25:57.339 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T21:25:57.340 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:25:57.340 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-17T21:25:57.341 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T21:25:57.341 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:25:57.341 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:25:57.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:25:57.363 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:25:57.360+0000 7ff4c08bb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:25:57.364 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:25:57.360+0000 7ff4c08bb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:25:57.365 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:25:57.360+0000 7ff4c08bb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:25:57.547 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:880: activate_osd: return 0 2026-02-17T21:25:57.547 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:353: TEST_divergent_ec: sleep 5 2026-02-17T21:25:58.608 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:25:58.604+0000 7ff4c08bb8c0 -1 Falling back to public interface 2026-02-17T21:26:01.604 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:01.600+0000 7ff4c08bb8c0 -1 osd.1 28 log_to_monitors true 2026-02-17T21:26:02.548 INFO:tasks.workunit.client.0.trial031.stdout:delay recovery divergent 2026-02-17T21:26:02.548 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:355: TEST_divergent_ec: echo 'delay recovery divergent' 2026-02-17T21:26:02.548 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:356: TEST_divergent_ec: ceph pg dump pgs 2026-02-17T21:26:02.747 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:26:02.748 INFO:tasks.workunit.client.0.trial031.stdout:1.0 101 0 101 0 0 71567 0 0 18 84 18 active+undersized+degraded 2026-02-17T21:25:24.385137+0000 41'102 52:145 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-17T21:24:00.509416+0000 0'0 2026-02-17T21:24:00.509416+0000 0 0 periodic scrub scheduled @ 2026-02-19T07:56:35.097793+0000 0 0 2026-02-17T21:26:02.748 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:26:02.748 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:26:02.749 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:26:02.749 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: get_asok_path osd.1 2026-02-17T21:26:02.749 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T21:26:02.749 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T21:26:02.750 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:26:02.750 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:26:02.750 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:26:02.750 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.16076/ceph-osd.1.asok 2026-02-17T21:26:02.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: CEPH_ARGS= 2026-02-17T21:26:02.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.16076/ceph-osd.1.asok set_recovery_delay 100000 2026-02-17T21:26:02.801 INFO:tasks.workunit.client.0.trial031.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-17T21:26:02.807 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:359: TEST_divergent_ec: ceph osd unset noup 2026-02-17T21:26:03.120 INFO:tasks.workunit.client.0.trial031.stderr:noup is unset 2026-02-17T21:26:03.131 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:361: TEST_divergent_ec: wait_for_osd up 0 2026-02-17T21:26:03.131 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:26:03.131 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:26:03.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:26:03.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:26:03.132 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:03.132 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:26:03.133 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:26:03.133 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:03.133 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:26:03.332 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 44 up_thru 48 down_at 43 last_clean_interval [35,41) [v2:127.0.0.1:6802/4003928214,v1:127.0.0.1:6803/4003928214] [v2:127.0.0.1:6804/4003928214,v1:127.0.0.1:6805/4003928214] exists,up 27052348-33c5-4b28-9883-30bfd1415320 2026-02-17T21:26:03.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:26:03.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:26:03.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:26:03.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:362: TEST_divergent_ec: wait_for_osd up 1 2026-02-17T21:26:03.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:26:03.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:26:03.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:26:03.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:26:03.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:03.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:26:03.335 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:26:03.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:03.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:26:03.530 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 54 up_thru 54 down_at 30 last_clean_interval [10,28) [v2:127.0.0.1:6818/1044942726,v1:127.0.0.1:6819/1044942726] [v2:127.0.0.1:6820/1044942726,v1:127.0.0.1:6821/1044942726] exists,up 7650e991-0aa1-446e-810d-5162931dd850 2026-02-17T21:26:03.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:26:03.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:26:03.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:26:03.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:363: TEST_divergent_ec: wait_for_osd up 2 2026-02-17T21:26:03.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:26:03.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:26:03.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:26:03.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:26:03.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:03.533 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:26:03.533 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:26:03.533 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:03.534 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:26:03.736 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 48 up_thru 0 down_at 47 last_clean_interval [37,41) [v2:127.0.0.1:6810/3527174734,v1:127.0.0.1:6811/3527174734] [v2:127.0.0.1:6812/3527174734,v1:127.0.0.1:6813/3527174734] exists,up a63bbb72-c70c-48fa-b386-d5e9b16de7b4 2026-02-17T21:26:03.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:26:03.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:26:03.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:26:03.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:365: TEST_divergent_ec: ceph pg dump pgs 2026-02-17T21:26:03.926 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:26:03.926 INFO:tasks.workunit.client.0.trial031.stdout:1.0 101 0 101 0 0 71567 0 0 18 84 18 active+undersized+degraded 2026-02-17T21:25:24.385137+0000 41'102 52:145 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-17T21:24:00.509416+0000 0'0 2026-02-17T21:24:00.509416+0000 0 0 periodic scrub scheduled @ 2026-02-19T07:56:35.097793+0000 0 0 2026-02-17T21:26:03.926 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:26:03.926 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:26:03.927 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:26:03.935 INFO:tasks.workunit.client.0.trial031.stdout:wait for peering 2026-02-17T21:26:03.935 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:366: TEST_divergent_ec: echo 'wait for peering' 2026-02-17T21:26:03.935 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:367: TEST_divergent_ec: ceph pg dump pgs 2026-02-17T21:26:04.120 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:26:04.120 INFO:tasks.workunit.client.0.trial031.stdout:1.0 101 0 101 0 0 71567 0 0 18 84 18 active+undersized+degraded 2026-02-17T21:25:24.385137+0000 41'102 52:145 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-17T21:24:00.509416+0000 0'0 2026-02-17T21:24:00.509416+0000 0 0 periodic scrub scheduled @ 2026-02-19T07:56:35.097793+0000 0 0 2026-02-17T21:26:04.120 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:26:04.120 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:26:04.120 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:26:04.128 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:368: TEST_divergent_ec: rados -p test put foo /etc/fstab 2026-02-17T21:26:04.158 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:370: TEST_divergent_ec: echo 'killing divergent 1' 2026-02-17T21:26:04.158 INFO:tasks.workunit.client.0.trial031.stdout:killing divergent 1 2026-02-17T21:26:04.158 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:371: TEST_divergent_ec: ceph pg dump pgs 2026-02-17T21:26:04.346 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:26:04.347 INFO:tasks.workunit.client.0.trial031.stdout:1.0 101 0 101 0 0 71567 0 0 18 84 18 active+undersized+degraded 2026-02-17T21:25:24.385137+0000 41'102 52:145 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-17T21:24:00.509416+0000 0'0 2026-02-17T21:24:00.509416+0000 0 0 periodic scrub scheduled @ 2026-02-19T07:56:35.097793+0000 0 0 2026-02-17T21:26:04.347 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:26:04.347 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:26:04.347 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:26:04.355 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:372: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.1 2026-02-17T21:26:04.356 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:26:04.356 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:26:04.357 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:26:04.357 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:26:04.357 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:26:04.464 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:26:04.464 INFO:tasks.workunit.client.0.trial031.stdout:reviving divergent 1 2026-02-17T21:26:04.465 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:374: TEST_divergent_ec: echo 'reviving divergent 1' 2026-02-17T21:26:04.465 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:375: TEST_divergent_ec: ceph pg dump pgs 2026-02-17T21:26:04.657 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:26:04.658 INFO:tasks.workunit.client.0.trial031.stdout:1.0 101 0 101 0 0 71567 0 0 18 84 18 active+undersized+degraded 2026-02-17T21:25:24.385137+0000 41'102 52:145 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-17T21:24:00.509416+0000 0'0 2026-02-17T21:24:00.509416+0000 0 0 periodic scrub scheduled @ 2026-02-19T07:56:35.097793+0000 0 0 2026-02-17T21:26:04.658 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:26:04.658 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:26:04.658 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:26:04.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:376: TEST_divergent_ec: activate_osd td/divergent-priors 1 2026-02-17T21:26:04.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-17T21:26:04.669 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:26:04.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T21:26:04.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:26:04.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-17T21:26:04.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-17T21:26:04.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:26:04.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:26:04.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:26:04.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-17T21:26:04.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-17T21:26:04.671 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:26:04.671 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:26:04.671 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-17T21:26:04.671 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:26:04.671 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:26:04.671 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:26:04.672 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:26:04.672 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:26:04.672 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:26:04.672 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:26:04.672 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' 2026-02-17T21:26:04.672 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:26:04.673 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-17T21:26:04.673 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-17T21:26:04.673 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:26:04.673 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:26:04.673 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:26:04.673 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:26:04.673 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:26:04.673 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:26:04.674 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-17T21:26:04.674 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:26:04.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T21:26:04.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=5ebac6e7-5321-4893-ad68-c06127ed455b --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.16076/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:26:04.676 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-17T21:26:04.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T21:26:04.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:26:04.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:26:04.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:26:04.694 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:04.692+0000 7f607fc2d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:04.696 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:04.692+0000 7f607fc2d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:04.697 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:04.692+0000 7f607fc2d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:04.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T21:26:04.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:26:04.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:26:04.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:26:04.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:26:04.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:04.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:26:04.875 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:26:04.875 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:04.875 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:26:05.082 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 54 up_thru 54 down_at 30 last_clean_interval [10,28) [v2:127.0.0.1:6818/1044942726,v1:127.0.0.1:6819/1044942726] [v2:127.0.0.1:6820/1044942726,v1:127.0.0.1:6821/1044942726] exists,up 7650e991-0aa1-446e-810d-5162931dd850 2026-02-17T21:26:05.083 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:26:05.083 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:26:05.083 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:26:05.083 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:378: TEST_divergent_ec: sleep 20 2026-02-17T21:26:05.964 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:05.960+0000 7f607fc2d8c0 -1 Falling back to public interface 2026-02-17T21:26:08.932 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:08.928+0000 7f607fc2d8c0 -1 osd.1 56 log_to_monitors true 2026-02-17T21:26:25.083 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:380: TEST_divergent_ec: echo 'allowing recovery' 2026-02-17T21:26:25.083 INFO:tasks.workunit.client.0.trial031.stdout:allowing recovery 2026-02-17T21:26:25.083 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:381: TEST_divergent_ec: ceph pg dump pgs 2026-02-17T21:26:25.274 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:26:25.274 INFO:tasks.workunit.client.0.trial031.stdout:1.0 102 0 0 0 0 72273 0 0 13 90 13 active+clean 2026-02-17T21:26:09.347071+0000 56'103 60:160 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:24:00.509416+0000 0'0 2026-02-17T21:24:00.509416+0000 0 0 periodic scrub scheduled @ 2026-02-19T07:55:00.191508+0000 0 0 2026-02-17T21:26:25.274 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:26:25.275 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:26:25.275 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:26:25.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:383: TEST_divergent_ec: for i in $osds 2026-02-17T21:26:25.283 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:385: TEST_divergent_ec: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-17T21:26:25.368 INFO:tasks.workunit.client.0.trial031.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-17T21:26:25.378 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:383: TEST_divergent_ec: for i in $osds 2026-02-17T21:26:25.378 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:385: TEST_divergent_ec: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-17T21:26:25.449 INFO:tasks.workunit.client.0.trial031.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-17T21:26:25.456 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:383: TEST_divergent_ec: for i in $osds 2026-02-17T21:26:25.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:385: TEST_divergent_ec: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-17T21:26:25.536 INFO:tasks.workunit.client.0.trial031.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-17T21:26:25.543 INFO:tasks.workunit.client.0.trial031.stdout:reading divergent objects 2026-02-17T21:26:25.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:388: TEST_divergent_ec: echo 'reading divergent objects' 2026-02-17T21:26:25.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:389: TEST_divergent_ec: ceph pg dump pgs 2026-02-17T21:26:25.742 INFO:tasks.workunit.client.0.trial031.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T21:26:25.742 INFO:tasks.workunit.client.0.trial031.stdout:1.0 102 0 0 0 0 72273 0 0 13 90 13 active+clean 2026-02-17T21:26:09.347071+0000 56'103 60:160 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T21:24:00.509416+0000 0'0 2026-02-17T21:24:00.509416+0000 0 0 periodic scrub scheduled @ 2026-02-19T07:55:00.191508+0000 0 0 2026-02-17T21:26:25.743 INFO:tasks.workunit.client.0.trial031.stdout: 2026-02-17T21:26:25.743 INFO:tasks.workunit.client.0.trial031.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T21:26:25.743 INFO:tasks.workunit.client.0.trial031.stderr:dumped pgs 2026-02-17T21:26:25.752 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: expr 2 + 2 2026-02-17T21:26:25.753 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: seq 1 4 2026-02-17T21:26:25.754 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:26:25.754 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_1 td/divergent-priors/existing 2026-02-17T21:26:25.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:26:25.775 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_2 td/divergent-priors/existing 2026-02-17T21:26:25.801 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:26:25.801 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_3 td/divergent-priors/existing 2026-02-17T21:26:25.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-17T21:26:25.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_4 td/divergent-priors/existing 2026-02-17T21:26:25.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:394: TEST_divergent_ec: rm -f td/divergent-priors/existing 2026-02-17T21:26:25.857 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:396: TEST_divergent_ec: find td/divergent-priors -name '*osd*log' 2026-02-17T21:26:25.859 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:396: TEST_divergent_ec: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-17T21:26:25.862 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 20 _merge_object_divergent_entries: merging hoid 1:7a80c0d5:::existing_99:1 entries: 28'103 (27'99) clone 1:7a80c0d5:::existing_99:1 by unknown.0.0:0 2026-02-17T21:24:13.458180+0000 0 snaps [1] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-17T21:26:25.863 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 20 _merge_object_divergent_entries: keeping 28'103 (27'99) clone 1:7a80c0d5:::existing_99:1 by unknown.0.0:0 2026-02-17T21:24:13.458180+0000 0 snaps [1] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-17T21:26:25.863 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-17T21:26:25.863 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 prior_version: 27'99 first_divergent_update: 28'103 last_divergent_update: 28'103 2026-02-17T21:26:25.863 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 has no more recent entries in log 2026-02-17T21:26:25.863 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 prior_version or op type indicates creation, deleting 2026-02-17T21:26:25.863 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 20 _merge_object_divergent_entries: merging hoid 1:7a80c0d5:::existing_99:head entries: 28'104 (27'99) modify 1:7a80c0d5:::existing_99:head by client.4581.0:1 2026-02-17T21:24:18.202163+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: true, new_object: false 2026-02-17T21:26:25.863 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 20 _merge_object_divergent_entries: keeping 28'104 (27'99) modify 1:7a80c0d5:::existing_99:head by client.4581.0:1 2026-02-17T21:24:18.202163+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: true, new_object: false 2026-02-17T21:26:25.863 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-17T21:26:25.863 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head prior_version: 27'99 first_divergent_update: 28'104 last_divergent_update: 28'104 2026-02-17T21:26:25.863 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head has no more recent entries in log 2026-02-17T21:26:25.863 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head must be rolled back or recovered, attempting to rollback 2026-02-17T21:26:25.864 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head olog_can_rollback_to: 28'104 2026-02-17T21:26:25.864 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head cannot rollback 28'104 (27'99) modify 1:7a80c0d5:::existing_99:head by client.4581.0:1 2026-02-17T21:24:18.202163+0000 0 ObjectCleanRegions clean_offsets: [(706, 18446744073709550909)], clean_omap: true, new_object: false 2026-02-17T21:26:25.864 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head cannot roll back, removing and adding to missing 2026-02-17T21:26:25.864 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 20 _merge_object_divergent_entries: merging hoid 1:c156033e:::existing_100:head entries: 27'101 (27'100) modify 1:c156033e:::existing_100:head by client.4572.0:1 2026-02-17T21:24:17.133121+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false,27'102 (27'101) modify 1:c156033e:::existing_100:head by client.4577.0:1 2026-02-17T21:24:18.129983+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:26:25.864 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 20 _merge_object_divergent_entries: keeping 27'101 (27'100) modify 1:c156033e:::existing_100:head by client.4572.0:1 2026-02-17T21:24:17.133121+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:26:25.865 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 20 _merge_object_divergent_entries: keeping 27'102 (27'101) modify 1:c156033e:::existing_100:head by client.4577.0:1 2026-02-17T21:24:18.129983+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:26:25.865 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-17T21:26:25.865 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head prior_version: 27'100 first_divergent_update: 27'101 last_divergent_update: 27'102 2026-02-17T21:26:25.865 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head has no more recent entries in log 2026-02-17T21:26:25.865 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head must be rolled back or recovered, attempting to rollback 2026-02-17T21:26:25.865 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head olog_can_rollback_to: 28'104 2026-02-17T21:26:25.865 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot rollback 27'102 (27'101) modify 1:c156033e:::existing_100:head by client.4577.0:1 2026-02-17T21:24:18.129983+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-17T21:26:25.865 INFO:tasks.workunit.client.0.trial031.stdout:td/divergent-priors/osd.1.log:2026-02-17T21:26:03.352+0000 7ff49e839640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot roll back, removing and adding to missing 2026-02-17T21:26:25.866 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:405: TEST_divergent_ec: find td/divergent-priors -name '*osd*log' 2026-02-17T21:26:25.866 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:405: TEST_divergent_ec: grep -q '_merge_object_divergent_entries.*rolled back' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-17T21:26:25.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:410: TEST_divergent_ec: echo success 2026-02-17T21:26:25.869 INFO:tasks.workunit.client.0.trial031.stdout:success 2026-02-17T21:26:25.870 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:412: TEST_divergent_ec: delete_pool test 2026-02-17T21:26:25.870 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=test 2026-02-17T21:26:25.870 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-17T21:26:26.124 INFO:tasks.workunit.client.0.trial031.stderr:pool 'test' does not exist 2026-02-17T21:26:26.133 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:413: TEST_divergent_ec: kill_daemons td/divergent-priors 2026-02-17T21:26:26.134 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:26:26.134 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:26:26.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:26:26.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:26:26.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:26:34.451 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:26:34.452 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-17T21:26:34.452 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-17T21:26:34.452 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:26:34.452 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-17T21:26:34.453 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:26:34.453 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:26:34.453 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:26:34.453 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:26:34.453 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:26:34.460 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:26:34.460 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:26:34.461 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:26:34.461 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:26:34.463 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:26:34.463 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:26:34.463 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:26:34.464 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:26:34.464 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:26:34.465 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:26:34.465 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:26:34.466 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:26:34.467 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:26:34.467 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-17T21:26:34.491 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:26:34.491 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:26:34.492 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:26:34.492 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.16076 2026-02-17T21:26:34.493 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:26:34.493 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:26:34.494 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T21:26:34.494 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/divergent-priors 0 2026-02-17T21:26:34.494 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-17T21:26:34.494 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T21:26:34.494 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-17T21:26:34.494 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:26:34.494 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:26:34.494 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:26:34.494 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:26:34.494 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:26:34.495 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:26:34.496 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:26:34.497 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:26:34.497 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:26:34.499 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:26:34.499 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:26:34.499 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:26:34.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:26:34.501 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:26:34.501 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:26:34.501 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:26:34.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:26:34.503 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T21:26:34.503 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-17T21:26:34.505 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:26:34.505 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:26:34.505 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.16076 2026-02-17T21:26:34.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.16076 2026-02-17T21:26:34.506 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:26:34.506 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:26:34.506 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T21:26:34.507 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T21:26:34.508 DEBUG:teuthology.orchestra.run.trial031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T21:26:34.562 INFO:tasks.workunit:Running workunit osd/ec-error-rollforward.sh... 2026-02-17T21:26:34.563 DEBUG:teuthology.orchestra.run.trial031:workunit test osd/ec-error-rollforward.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d855f53b89fdcec760fd9232a5fb55ed4fb111a1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh 2026-02-17T21:26:34.616 INFO:tasks.workunit.client.0.trial031.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T21:26:34.619 INFO:tasks.workunit.client.0.trial031.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T21:26:34.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T21:26:34.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T21:26:34.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T21:26:34.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T21:26:34.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T21:26:34.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T21:26:34.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T21:26:34.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T21:26:34.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/ec-error-rollforward 2026-02-17T21:26:34.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:6: run: local dir=td/ec-error-rollforward 2026-02-17T21:26:34.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:7: run: shift 2026-02-17T21:26:34.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:10: run: export CEPH_MON=127.0.0.1:7132 2026-02-17T21:26:34.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:10: run: CEPH_MON=127.0.0.1:7132 2026-02-17T21:26:34.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:11: run: export CEPH_ARGS 2026-02-17T21:26:34.620 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:12: run: uuidgen 2026-02-17T21:26:34.621 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:12: run: CEPH_ARGS+='--fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none ' 2026-02-17T21:26:34.621 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:13: run: CEPH_ARGS+='--mon-host=127.0.0.1:7132 ' 2026-02-17T21:26:34.621 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:14: run: export margin=10 2026-02-17T21:26:34.621 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:14: run: margin=10 2026-02-17T21:26:34.621 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:15: run: export objects=200 2026-02-17T21:26:34.622 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:15: run: objects=200 2026-02-17T21:26:34.622 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:16: run: export poolname=test 2026-02-17T21:26:34.622 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:16: run: poolname=test 2026-02-17T21:26:34.622 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: set 2026-02-17T21:26:34.622 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T21:26:34.623 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: local funcs=TEST_ec_error_rollforward 2026-02-17T21:26:34.624 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:19: run: for func in $funcs 2026-02-17T21:26:34.624 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:20: run: setup td/ec-error-rollforward 2026-02-17T21:26:34.624 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/ec-error-rollforward 2026-02-17T21:26:34.624 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/ec-error-rollforward 2026-02-17T21:26:34.624 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/ec-error-rollforward 2026-02-17T21:26:34.624 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:26:34.625 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/ec-error-rollforward KILL 2026-02-17T21:26:34.625 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:26:34.625 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:26:34.625 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:26:34.625 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:26:34.625 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:26:34.626 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:26:34.626 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:26:34.627 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:26:34.627 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:26:34.628 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:26:34.628 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:26:34.628 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:26:34.630 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:26:34.630 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:26:34.630 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:26:34.630 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:26:34.631 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:26:34.632 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:26:34.632 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/ec-error-rollforward 2026-02-17T21:26:34.633 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:26:34.633 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:26:34.633 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:26:34.633 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.74709 2026-02-17T21:26:34.634 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:26:34.634 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:26:34.634 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/ec-error-rollforward 2026-02-17T21:26:34.636 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T21:26:34.636 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:26:34.636 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:26:34.636 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.74709 2026-02-17T21:26:34.637 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T21:26:34.639 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T21:26:34.639 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T21:26:34.639 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T21:26:34.639 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/ec-error-rollforward 1' TERM HUP INT 2026-02-17T21:26:34.639 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:21: run: TEST_ec_error_rollforward td/ec-error-rollforward 2026-02-17T21:26:34.639 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:27: TEST_ec_error_rollforward: local dir=td/ec-error-rollforward 2026-02-17T21:26:34.639 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:28: TEST_ec_error_rollforward: run_mon td/ec-error-rollforward a 2026-02-17T21:26:34.640 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/ec-error-rollforward 2026-02-17T21:26:34.640 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T21:26:34.640 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T21:26:34.640 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T21:26:34.640 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/ec-error-rollforward/a 2026-02-17T21:26:34.640 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/ec-error-rollforward/a --run-dir=td/ec-error-rollforward 2026-02-17T21:26:34.659 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T21:26:34.659 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:26:34.659 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:26:34.659 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:26:34.660 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:26:34.660 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:26:34.660 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:26:34.660 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ec-error-rollforward/a '--log-file=td/ec-error-rollforward/$name.log' '--admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' --mon-cluster-log-file=td/ec-error-rollforward/log --run-dir=td/ec-error-rollforward '--pid-file=td/ec-error-rollforward/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T21:26:34.686 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T21:26:34.687 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T21:26:34.687 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:26:34.687 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:26:34.687 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T21:26:34.687 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T21:26:34.688 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:26:34.688 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:26:34.688 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:26:34.689 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:26:34.689 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:26:34.689 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:26:34.689 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.74709/ceph-mon.a.asok 2026-02-17T21:26:34.689 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:26:34.689 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.74709/ceph-mon.a.asok config get fsid 2026-02-17T21:26:34.743 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T21:26:34.743 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:26:34.743 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:26:34.743 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T21:26:34.743 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T21:26:34.743 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:26:34.743 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:26:34.743 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:26:34.744 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:26:34.744 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:26:34.744 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:26:34.744 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.74709/ceph-mon.a.asok 2026-02-17T21:26:34.744 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:26:34.745 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.74709/ceph-mon.a.asok config get mon_host 2026-02-17T21:26:34.797 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:29: TEST_ec_error_rollforward: run_mgr td/ec-error-rollforward x 2026-02-17T21:26:34.797 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/ec-error-rollforward 2026-02-17T21:26:34.797 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T21:26:34.797 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T21:26:34.797 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T21:26:34.797 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/ec-error-rollforward/x 2026-02-17T21:26:34.798 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T21:26:34.903 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T21:26:34.903 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:26:34.903 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:26:34.903 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:26:34.903 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:26:34.903 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:26:34.903 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:26:34.904 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:26:34.904 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ec-error-rollforward/x '--log-file=td/ec-error-rollforward/$name.log' '--admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' --run-dir=td/ec-error-rollforward '--pid-file=td/ec-error-rollforward/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:26:34.921 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:30: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 0 2026-02-17T21:26:34.921 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/ec-error-rollforward 2026-02-17T21:26:34.921 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:26:34.921 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T21:26:34.921 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:26:34.921 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/ec-error-rollforward/0 2026-02-17T21:26:34.921 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-17T21:26:34.922 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:26:34.922 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:26:34.922 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:26:34.922 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/0' 2026-02-17T21:26:34.922 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/0/journal' 2026-02-17T21:26:34.922 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:26:34.922 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:26:34.922 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-17T21:26:34.922 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:26:34.922 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:26:34.923 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:26:34.923 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:26:34.923 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:26:34.923 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:26:34.923 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:26:34.923 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:26:34.923 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:26:34.923 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:26:34.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:26:34.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-17T21:26:34.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-17T21:26:34.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:26:34.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:26:34.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:26:34.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:26:34.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:26:34.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:26:34.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/ec-error-rollforward/0 2026-02-17T21:26:34.924 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:26:34.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=72921bf3-6f61-4b2d-bd6f-977cd5885a86 2026-02-17T21:26:34.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 72921bf3-6f61-4b2d-bd6f-977cd5885a86' 2026-02-17T21:26:34.925 INFO:tasks.workunit.client.0.trial031.stdout:add osd0 72921bf3-6f61-4b2d-bd6f-977cd5885a86 2026-02-17T21:26:34.925 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:26:34.935 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAK3ZRpO7DUNxAA9G/rhGEUvB8ujSrDKpnLHw== 2026-02-17T21:26:34.935 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAK3ZRpO7DUNxAA9G/rhGEUvB8ujSrDKpnLHw=="}' 2026-02-17T21:26:34.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 72921bf3-6f61-4b2d-bd6f-977cd5885a86 -i td/ec-error-rollforward/0/new.json 2026-02-17T21:26:35.047 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:26:35.055 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/ec-error-rollforward/0/new.json 2026-02-17T21:26:35.056 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAK3ZRpO7DUNxAA9G/rhGEUvB8ujSrDKpnLHw== --osd-uuid 72921bf3-6f61-4b2d-bd6f-977cd5885a86 2026-02-17T21:26:35.071 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:35.071+0000 7fbf49bc28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:35.072 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:35.071+0000 7fbf49bc28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:35.073 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:35.071+0000 7fbf49bc28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:35.074 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:35.071+0000 7fbf49bc28c0 -1 bdev(0x55a2ac5e1c00 td/ec-error-rollforward/0/block) open stat got: (1) Operation not permitted 2026-02-17T21:26:35.074 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:35.071+0000 7fbf49bc28c0 -1 bluestore(td/ec-error-rollforward/0) _read_fsid unparsable uuid 2026-02-17T21:26:37.851 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/ec-error-rollforward/0/keyring 2026-02-17T21:26:37.851 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:26:37.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T21:26:37.853 INFO:tasks.workunit.client.0.trial031.stdout:adding osd0 key to auth repository 2026-02-17T21:26:37.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/ec-error-rollforward/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:26:38.120 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:26:38.121 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T21:26:38.121 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:26:38.121 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:26:38.121 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:26:38.121 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:26:38.141 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:38.139+0000 7f2a435f68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:38.143 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:38.143+0000 7f2a435f68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:38.144 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:38.143+0000 7f2a435f68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:38.325 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T21:26:38.325 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:26:38.325 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:26:38.325 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:26:38.325 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:26:38.325 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:38.326 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:26:38.327 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:26:38.327 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:38.327 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:26:38.519 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:26:39.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:26:39.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:39.521 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:26:39.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:26:39.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:39.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:26:39.632 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:39.631+0000 7f2a435f68c0 -1 Falling back to public interface 2026-02-17T21:26:39.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:26:40.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:26:40.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:40.719 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:26:40.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:26:40.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:40.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:26:40.915 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:26:41.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:26:41.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:41.918 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:26:41.918 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:26:41.918 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:41.918 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:26:42.111 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:26:42.770 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:42.767+0000 7f2a435f68c0 -1 osd.0 0 log_to_monitors true 2026-02-17T21:26:43.113 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:26:43.113 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:43.113 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:26:43.113 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:26:43.114 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:43.114 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:26:43.313 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:26:44.314 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:26:44.314 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:44.315 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:26:44.315 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:26:44.315 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:44.315 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:26:44.508 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2547266924,v1:127.0.0.1:6803/2547266924] [v2:127.0.0.1:6804/2547266924,v1:127.0.0.1:6805/2547266924] exists,up 72921bf3-6f61-4b2d-bd6f-977cd5885a86 2026-02-17T21:26:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:26:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:26:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:26:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:31: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 1 2026-02-17T21:26:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/ec-error-rollforward 2026-02-17T21:26:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:26:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T21:26:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:26:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/ec-error-rollforward/1 2026-02-17T21:26:44.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-17T21:26:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:26:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:26:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:26:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/1' 2026-02-17T21:26:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/1/journal' 2026-02-17T21:26:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:26:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:26:44.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-17T21:26:44.513 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:26:44.513 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:26:44.513 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:26:44.513 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:26:44.514 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:26:44.514 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:26:44.514 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:26:44.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:26:44.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:26:44.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:26:44.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:26:44.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-17T21:26:44.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-17T21:26:44.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:26:44.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:26:44.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:26:44.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:26:44.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:26:44.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:26:44.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/ec-error-rollforward/1 2026-02-17T21:26:44.515 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:26:44.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=8429fe43-a911-43ac-b1a0-82fb9024585d 2026-02-17T21:26:44.516 INFO:tasks.workunit.client.0.trial031.stdout:add osd1 8429fe43-a911-43ac-b1a0-82fb9024585d 2026-02-17T21:26:44.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 8429fe43-a911-43ac-b1a0-82fb9024585d' 2026-02-17T21:26:44.516 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:26:44.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAU3ZRpW3FQHxAA1TAf+8Us40Jelm/Y9w3OOQ== 2026-02-17T21:26:44.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAU3ZRpW3FQHxAA1TAf+8Us40Jelm/Y9w3OOQ=="}' 2026-02-17T21:26:44.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 8429fe43-a911-43ac-b1a0-82fb9024585d -i td/ec-error-rollforward/1/new.json 2026-02-17T21:26:44.726 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:26:44.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/ec-error-rollforward/1/new.json 2026-02-17T21:26:44.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAU3ZRpW3FQHxAA1TAf+8Us40Jelm/Y9w3OOQ== --osd-uuid 8429fe43-a911-43ac-b1a0-82fb9024585d 2026-02-17T21:26:44.751 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:44.751+0000 7f75bf9c28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:44.752 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:44.751+0000 7f75bf9c28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:44.753 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:44.751+0000 7f75bf9c28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:44.754 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:44.751+0000 7f75bf9c28c0 -1 bdev(0x55c743d4dc00 td/ec-error-rollforward/1/block) open stat got: (1) Operation not permitted 2026-02-17T21:26:44.754 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:44.751+0000 7f75bf9c28c0 -1 bluestore(td/ec-error-rollforward/1) _read_fsid unparsable uuid 2026-02-17T21:26:47.271 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/ec-error-rollforward/1/keyring 2026-02-17T21:26:47.271 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:26:47.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T21:26:47.272 INFO:tasks.workunit.client.0.trial031.stdout:adding osd1 key to auth repository 2026-02-17T21:26:47.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/ec-error-rollforward/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:26:47.542 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:26:47.542 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T21:26:47.542 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:26:47.542 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:26:47.542 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:26:47.542 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:26:47.557 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:47.555+0000 7fe2db5b98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:47.558 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:47.559+0000 7fe2db5b98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:47.559 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:47.559+0000 7fe2db5b98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:47.749 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T21:26:47.749 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:26:47.749 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:26:47.749 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:26:47.749 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:26:47.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:47.750 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:26:47.751 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:26:47.751 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:47.751 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:26:47.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:26:48.947 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:26:48.947 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:48.948 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:26:48.948 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:26:48.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:48.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:26:49.059 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:49.059+0000 7fe2db5b98c0 -1 Falling back to public interface 2026-02-17T21:26:49.140 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:26:50.142 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:26:50.142 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:50.142 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:26:50.143 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:26:50.143 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:50.143 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:26:50.340 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:26:51.341 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:26:51.341 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:51.341 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:26:51.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:26:51.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:51.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:26:51.537 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:26:52.098 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:52.095+0000 7fe2db5b98c0 -1 osd.1 0 log_to_monitors true 2026-02-17T21:26:52.538 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:26:52.538 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:52.539 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:26:52.539 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:26:52.539 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:52.540 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:26:52.733 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1769007844,v1:127.0.0.1:6811/1769007844] [v2:127.0.0.1:6812/1769007844,v1:127.0.0.1:6813/1769007844] exists,up 8429fe43-a911-43ac-b1a0-82fb9024585d 2026-02-17T21:26:52.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:26:52.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:26:52.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:26:52.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:32: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 2 2026-02-17T21:26:52.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/ec-error-rollforward 2026-02-17T21:26:52.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:26:52.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T21:26:52.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:26:52.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/ec-error-rollforward/2 2026-02-17T21:26:52.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-17T21:26:52.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:26:52.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:26:52.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:26:52.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/2' 2026-02-17T21:26:52.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/2/journal' 2026-02-17T21:26:52.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:26:52.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:26:52.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-17T21:26:52.737 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:26:52.738 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:26:52.738 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:26:52.738 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:26:52.738 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:26:52.738 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:26:52.738 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:26:52.738 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:26:52.738 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:26:52.738 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:26:52.738 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:26:52.738 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-17T21:26:52.738 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-17T21:26:52.738 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:26:52.739 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:26:52.739 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:26:52.739 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:26:52.739 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:26:52.739 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:26:52.739 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/ec-error-rollforward/2 2026-02-17T21:26:52.740 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:26:52.740 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=8821472b-9e09-4b68-8e2c-05766972b460 2026-02-17T21:26:52.740 INFO:tasks.workunit.client.0.trial031.stdout:add osd2 8821472b-9e09-4b68-8e2c-05766972b460 2026-02-17T21:26:52.740 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 8821472b-9e09-4b68-8e2c-05766972b460' 2026-02-17T21:26:52.741 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:26:52.749 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAc3ZRpItCuLBAAh5hPLU4R6HIYNJAMo51O1g== 2026-02-17T21:26:52.749 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAc3ZRpItCuLBAAh5hPLU4R6HIYNJAMo51O1g=="}' 2026-02-17T21:26:52.749 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 8821472b-9e09-4b68-8e2c-05766972b460 -i td/ec-error-rollforward/2/new.json 2026-02-17T21:26:52.948 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:26:52.957 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/ec-error-rollforward/2/new.json 2026-02-17T21:26:52.958 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAc3ZRpItCuLBAAh5hPLU4R6HIYNJAMo51O1g== --osd-uuid 8821472b-9e09-4b68-8e2c-05766972b460 2026-02-17T21:26:52.973 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:52.971+0000 7f34c4ca08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:52.975 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:52.975+0000 7f34c4ca08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:52.976 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:52.975+0000 7f34c4ca08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:52.976 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:52.975+0000 7f34c4ca08c0 -1 bdev(0x55fbb1953c00 td/ec-error-rollforward/2/block) open stat got: (1) Operation not permitted 2026-02-17T21:26:52.977 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:52.975+0000 7f34c4ca08c0 -1 bluestore(td/ec-error-rollforward/2) _read_fsid unparsable uuid 2026-02-17T21:26:55.267 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/ec-error-rollforward/2/keyring 2026-02-17T21:26:55.267 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:26:55.268 INFO:tasks.workunit.client.0.trial031.stdout:adding osd2 key to auth repository 2026-02-17T21:26:55.269 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T21:26:55.269 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/ec-error-rollforward/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:26:55.533 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T21:26:55.533 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:26:55.534 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:26:55.534 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:26:55.534 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:26:55.534 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:26:55.554 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:55.555+0000 7f1a2c2d88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:55.556 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:55.555+0000 7f1a2c2d88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:55.557 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:55.555+0000 7f1a2c2d88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:26:55.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T21:26:55.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:26:55.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:26:55.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:26:55.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:26:55.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:55.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:26:55.735 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:26:55.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:55.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:26:55.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:26:56.932 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:26:56.932 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:56.933 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:26:56.934 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:26:56.934 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:56.934 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:26:57.032 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:26:57.030+0000 7f1a2c2d88c0 -1 Falling back to public interface 2026-02-17T21:26:57.126 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:26:58.128 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:26:58.128 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:58.129 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:26:58.129 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:26:58.129 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:58.129 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:26:58.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:26:59.333 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:26:59.333 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:26:59.333 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:26:59.333 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:26:59.333 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:26:59.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:26:59.527 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:00.240 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:00.238+0000 7f1a2c2d88c0 -1 osd.2 0 log_to_monitors true 2026-02-17T21:27:00.529 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:00.529 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:00.529 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:27:00.530 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:27:00.530 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:00.531 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:27:00.724 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:01.726 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:01.726 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:01.726 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:27:01.727 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:27:01.727 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:27:01.727 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:01.920 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2539623502,v1:127.0.0.1:6819/2539623502] [v2:127.0.0.1:6820/2539623502,v1:127.0.0.1:6821/2539623502] exists,up 8821472b-9e09-4b68-8e2c-05766972b460 2026-02-17T21:27:01.923 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:27:01.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:27:01.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:27:01.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:33: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 3 2026-02-17T21:27:01.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/ec-error-rollforward 2026-02-17T21:27:01.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:27:01.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T21:27:01.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:27:01.924 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/ec-error-rollforward/3 2026-02-17T21:27:01.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-17T21:27:01.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:27:01.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:27:01.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:27:01.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/3' 2026-02-17T21:27:01.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/3/journal' 2026-02-17T21:27:01.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:27:01.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:27:01.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-17T21:27:01.925 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:27:01.925 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:27:01.926 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:27:01.926 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:27:01.926 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:01.926 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:27:01.926 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:27:01.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:27:01.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:27:01.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:27:01.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:27:01.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-17T21:27:01.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-17T21:27:01.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:27:01.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:27:01.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:27:01.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:27:01.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:27:01.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:27:01.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/ec-error-rollforward/3 2026-02-17T21:27:01.927 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:27:01.928 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=8923027a-dd64-4ebe-b459-4becb8350035 2026-02-17T21:27:01.928 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 8923027a-dd64-4ebe-b459-4becb8350035' 2026-02-17T21:27:01.928 INFO:tasks.workunit.client.0.trial031.stdout:add osd3 8923027a-dd64-4ebe-b459-4becb8350035 2026-02-17T21:27:01.929 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:27:01.937 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAl3ZRpinrkNxAAe95fOP+m+eRYhOCyrZF1Dw== 2026-02-17T21:27:01.937 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAl3ZRpinrkNxAAe95fOP+m+eRYhOCyrZF1Dw=="}' 2026-02-17T21:27:01.937 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 8923027a-dd64-4ebe-b459-4becb8350035 -i td/ec-error-rollforward/3/new.json 2026-02-17T21:27:02.138 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:27:02.147 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/ec-error-rollforward/3/new.json 2026-02-17T21:27:02.148 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAl3ZRpinrkNxAAe95fOP+m+eRYhOCyrZF1Dw== --osd-uuid 8923027a-dd64-4ebe-b459-4becb8350035 2026-02-17T21:27:02.162 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:02.162+0000 7ffbd29268c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:02.164 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:02.162+0000 7ffbd29268c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:02.166 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:02.162+0000 7ffbd29268c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:02.166 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:02.162+0000 7ffbd29268c0 -1 bdev(0x555ad8c45c00 td/ec-error-rollforward/3/block) open stat got: (1) Operation not permitted 2026-02-17T21:27:02.166 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:02.162+0000 7ffbd29268c0 -1 bluestore(td/ec-error-rollforward/3) _read_fsid unparsable uuid 2026-02-17T21:27:04.464 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/ec-error-rollforward/3/keyring 2026-02-17T21:27:04.464 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:27:04.464 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T21:27:04.465 INFO:tasks.workunit.client.0.trial031.stdout:adding osd3 key to auth repository 2026-02-17T21:27:04.465 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/ec-error-rollforward/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:27:04.751 INFO:tasks.workunit.client.0.trial031.stdout:start osd.3 2026-02-17T21:27:04.752 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T21:27:04.752 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:27:04.752 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:27:04.753 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:27:04.753 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:27:04.765 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:04.762+0000 7fedbb2768c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:04.767 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:04.766+0000 7fedbb2768c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:04.768 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:04.766+0000 7fedbb2768c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:04.951 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T21:27:04.952 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:27:04.952 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T21:27:04.952 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:27:04.952 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:27:04.952 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:04.952 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:27:04.953 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:27:04.953 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:04.953 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:27:05.146 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:06.148 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:06.148 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:06.148 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:27:06.148 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:27:06.148 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:06.148 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:27:06.280 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:06.278+0000 7fedbb2768c0 -1 Falling back to public interface 2026-02-17T21:27:06.342 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:07.343 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:07.344 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:07.344 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:27:07.344 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:27:07.345 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:07.345 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:27:07.536 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:08.538 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:08.538 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:08.538 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:27:08.539 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:27:08.539 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:08.539 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:27:08.730 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:09.313 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:09.310+0000 7fedbb2768c0 -1 osd.3 0 log_to_monitors true 2026-02-17T21:27:09.732 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:09.732 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:09.732 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:27:09.732 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:27:09.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:09.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:27:09.935 INFO:tasks.workunit.client.0.trial031.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/227136352,v1:127.0.0.1:6827/227136352] [v2:127.0.0.1:6828/227136352,v1:127.0.0.1:6829/227136352] exists,up 8923027a-dd64-4ebe-b459-4becb8350035 2026-02-17T21:27:09.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:27:09.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:27:09.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:27:09.936 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:35: TEST_ec_error_rollforward: ceph osd erasure-code-profile set ec-profile m=2 k=2 crush-failure-domain=osd 2026-02-17T21:27:10.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:36: TEST_ec_error_rollforward: ceph osd pool create ec 1 1 erasure ec-profile 2026-02-17T21:27:10.509 INFO:tasks.workunit.client.0.trial031.stderr:pool 'ec' already exists 2026-02-17T21:27:10.518 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:38: TEST_ec_error_rollforward: rados -p ec put foo /etc/passwd 2026-02-17T21:27:11.480 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:40: TEST_ec_error_rollforward: cat td/ec-error-rollforward/osd.2.pid 2026-02-17T21:27:11.481 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:40: TEST_ec_error_rollforward: kill -STOP 79000 2026-02-17T21:27:11.482 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:43: TEST_ec_error_rollforward: pids=81303 2026-02-17T21:27:11.482 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:44: TEST_ec_error_rollforward: sleep 1 2026-02-17T21:27:11.482 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:42: TEST_ec_error_rollforward: rados -p ec rm foo 2026-02-17T21:27:12.484 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:46: TEST_ec_error_rollforward: pids+=' 81324' 2026-02-17T21:27:12.484 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:45: TEST_ec_error_rollforward: rados -p ec rm a 2026-02-17T21:27:12.484 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:48: TEST_ec_error_rollforward: pids+=' 81325' 2026-02-17T21:27:12.484 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:47: TEST_ec_error_rollforward: rados -p ec rm b 2026-02-17T21:27:12.484 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:50: TEST_ec_error_rollforward: pids+=' 81326' 2026-02-17T21:27:12.484 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:51: TEST_ec_error_rollforward: sleep 1 2026-02-17T21:27:12.484 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:49: TEST_ec_error_rollforward: rados -p ec rm c 2026-02-17T21:27:13.485 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:54: TEST_ec_error_rollforward: kill_daemons td/ec-error-rollforward KILL osd 2026-02-17T21:27:13.486 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:27:13.486 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:27:13.486 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:27:13.487 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:27:13.487 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:27:13.596 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:27:13.596 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:55: TEST_ec_error_rollforward: kill 81303 81324 81325 81326 2026-02-17T21:27:13.597 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:56: TEST_ec_error_rollforward: wait 2026-02-17T21:27:13.600 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:58: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 0 2026-02-17T21:27:13.600 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/ec-error-rollforward 2026-02-17T21:27:13.600 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:27:13.600 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T21:27:13.600 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:27:13.600 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/ec-error-rollforward/0 2026-02-17T21:27:13.600 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-17T21:27:13.600 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:27:13.601 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:27:13.601 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:27:13.601 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/0' 2026-02-17T21:27:13.601 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/0/journal' 2026-02-17T21:27:13.601 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:27:13.601 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:27:13.601 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-17T21:27:13.601 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:27:13.601 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:27:13.602 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:27:13.602 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:27:13.602 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:13.602 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:27:13.602 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:27:13.602 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:27:13.602 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:27:13.602 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-17T21:27:13.602 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-17T21:27:13.602 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:27:13.603 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:27:13.603 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:27:13.603 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:27:13.603 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:27:13.603 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:27:13.603 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/ec-error-rollforward/0 2026-02-17T21:27:13.604 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:27:13.606 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T21:27:13.606 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:27:13.606 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/ec-error-rollforward/0/whoami 2026-02-17T21:27:13.606 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T21:27:13.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:27:13.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:27:13.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:27:13.617 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:13.614+0000 7f721c2ce8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:13.619 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:13.618+0000 7f721c2ce8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:13.620 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:13.618+0000 7f721c2ce8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:13.803 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T21:27:13.803 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:27:13.803 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:27:13.803 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:27:13.803 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:27:13.803 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:13.804 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:27:13.804 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:27:13.805 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:13.805 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:27:13.996 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2547266924,v1:127.0.0.1:6803/2547266924] [v2:127.0.0.1:6804/2547266924,v1:127.0.0.1:6805/2547266924] exists,up 72921bf3-6f61-4b2d-bd6f-977cd5885a86 2026-02-17T21:27:14.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:27:14.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:27:14.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:27:14.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:59: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 1 2026-02-17T21:27:14.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/ec-error-rollforward 2026-02-17T21:27:14.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:27:14.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T21:27:14.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:27:14.000 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/ec-error-rollforward/1 2026-02-17T21:27:14.001 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-17T21:27:14.001 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:27:14.001 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:27:14.001 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:27:14.001 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/1' 2026-02-17T21:27:14.001 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/1/journal' 2026-02-17T21:27:14.001 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:27:14.001 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:27:14.001 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-17T21:27:14.001 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:27:14.002 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:27:14.002 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:27:14.002 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:27:14.002 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:14.002 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:27:14.002 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:27:14.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:27:14.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:27:14.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-17T21:27:14.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-17T21:27:14.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:27:14.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:27:14.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:27:14.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:27:14.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:27:14.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:27:14.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/ec-error-rollforward/1 2026-02-17T21:27:14.003 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:27:14.005 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T21:27:14.005 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:27:14.005 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/ec-error-rollforward/1/whoami 2026-02-17T21:27:14.005 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T21:27:14.005 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:27:14.005 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:27:14.005 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:27:14.013 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:14.010+0000 7f1ac80ae8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:14.015 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:14.014+0000 7f1ac80ae8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:14.016 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:14.014+0000 7f1ac80ae8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:14.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T21:27:14.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:27:14.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:27:14.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:27:14.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:27:14.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:14.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:27:14.204 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:27:14.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:14.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:27:14.400 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 9 up_thru 22 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1769007844,v1:127.0.0.1:6811/1769007844] [v2:127.0.0.1:6812/1769007844,v1:127.0.0.1:6813/1769007844] exists,up 8429fe43-a911-43ac-b1a0-82fb9024585d 2026-02-17T21:27:14.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:27:14.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:27:14.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:27:14.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:60: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 2 2026-02-17T21:27:14.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/ec-error-rollforward 2026-02-17T21:27:14.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:27:14.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T21:27:14.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:27:14.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/ec-error-rollforward/2 2026-02-17T21:27:14.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-17T21:27:14.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:27:14.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:27:14.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:27:14.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/2' 2026-02-17T21:27:14.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/2/journal' 2026-02-17T21:27:14.405 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:27:14.405 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:27:14.405 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-17T21:27:14.405 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:27:14.405 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:27:14.405 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:27:14.405 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:27:14.405 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:14.405 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:27:14.405 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:27:14.405 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:27:14.405 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:27:14.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-17T21:27:14.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-17T21:27:14.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:27:14.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:27:14.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:27:14.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:27:14.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:27:14.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:27:14.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/ec-error-rollforward/2 2026-02-17T21:27:14.406 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:27:14.408 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T21:27:14.408 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:27:14.408 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/ec-error-rollforward/2/whoami 2026-02-17T21:27:14.408 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T21:27:14.408 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:27:14.408 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:27:14.408 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:27:14.424 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:14.422+0000 7fcfcb70a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:14.425 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:14.422+0000 7fcfcb70a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:14.426 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:14.426+0000 7fcfcb70a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:14.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T21:27:14.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:27:14.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:27:14.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:27:14.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:27:14.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:14.608 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:27:14.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:27:14.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:14.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:27:14.805 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2539623502,v1:127.0.0.1:6819/2539623502] [v2:127.0.0.1:6820/2539623502,v1:127.0.0.1:6821/2539623502] exists,up 8821472b-9e09-4b68-8e2c-05766972b460 2026-02-17T21:27:14.809 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:27:14.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:27:14.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:27:14.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:61: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 3 2026-02-17T21:27:14.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/ec-error-rollforward 2026-02-17T21:27:14.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:27:14.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T21:27:14.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:27:14.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/ec-error-rollforward/3 2026-02-17T21:27:14.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-17T21:27:14.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:27:14.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:27:14.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:27:14.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/3' 2026-02-17T21:27:14.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/3/journal' 2026-02-17T21:27:14.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:27:14.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:27:14.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-17T21:27:14.811 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:27:14.811 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:27:14.811 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:27:14.811 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:27:14.812 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:14.812 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:27:14.812 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:27:14.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' 2026-02-17T21:27:14.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:27:14.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-17T21:27:14.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-17T21:27:14.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:27:14.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:27:14.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:27:14.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:27:14.813 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:27:14.813 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:27:14.813 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/ec-error-rollforward/3 2026-02-17T21:27:14.813 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T21:27:14.813 INFO:tasks.workunit.client.0.trial031.stdout:start osd.3 2026-02-17T21:27:14.814 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=2906030c-c64a-475e-8682-b780ded9943c --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.74709/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:27:14.814 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/ec-error-rollforward/3/whoami 2026-02-17T21:27:14.815 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T21:27:14.815 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:27:14.815 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:27:14.815 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:27:14.831 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:14.830+0000 7f82d961e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:14.832 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:14.830+0000 7f82d961e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:14.833 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:14.830+0000 7f82d961e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:14.899 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:14.898+0000 7f721c2ce8c0 -1 Falling back to public interface 2026-02-17T21:27:15.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T21:27:15.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:27:15.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T21:27:15.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:27:15.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:27:15.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:15.003 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:27:15.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:27:15.004 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:15.004 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:27:15.200 INFO:tasks.workunit.client.0.trial031.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/227136352,v1:127.0.0.1:6827/227136352] [v2:127.0.0.1:6828/227136352,v1:127.0.0.1:6829/227136352] exists,up 8923027a-dd64-4ebe-b459-4becb8350035 2026-02-17T21:27:15.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:27:15.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:27:15.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:27:15.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:63: TEST_ec_error_rollforward: wait_for_clean 2026-02-17T21:27:15.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:27:15.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:27:15.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:27:15.202 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T21:27:15.203 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:27:15.203 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:27:15.203 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:27:15.203 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:27:15.203 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:27:15.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T21:27:15.310 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:27:15.310 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:27:15.310 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:27:15.310 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:27:15.310 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:27:15.420 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:15.418+0000 7f1ac80ae8c0 -1 Falling back to public interface 2026-02-17T21:27:15.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:27:15.505 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:27:15.505 INFO:tasks.workunit.client.0.trial031.stderr:2 2026-02-17T21:27:15.505 INFO:tasks.workunit.client.0.trial031.stderr:3' 2026-02-17T21:27:15.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:27:15.505 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:27:15.506 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:27:15.716 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:15.714+0000 7fcfcb70a8c0 -1 Falling back to public interface 2026-02-17T21:27:16.376 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:16.374+0000 7f82d961e8c0 -1 Falling back to public interface 2026-02-17T21:27:17.997 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:17.994+0000 7f721c2ce8c0 -1 osd.0 24 log_to_monitors true 2026-02-17T21:27:18.553 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:18.550+0000 7f1ac80ae8c0 -1 osd.1 24 log_to_monitors true 2026-02-17T21:27:18.823 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:18.822+0000 7fcfcb70a8c0 -1 osd.2 23 log_to_monitors true 2026-02-17T21:27:19.466 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:19.466+0000 7f82d961e8c0 -1 osd.3 24 log_to_monitors true 2026-02-17T21:27:20.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964116995 2026-02-17T21:27:20.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964116995 2026-02-17T21:27:20.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-115964116995' 2026-02-17T21:27:20.592 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:27:20.592 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:27:20.663 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018883 2026-02-17T21:27:20.663 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018883 2026-02-17T21:27:20.663 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-115964116995 1-128849018883' 2026-02-17T21:27:20.663 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:27:20.663 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:27:20.739 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=141733920771 2026-02-17T21:27:20.739 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 141733920771 2026-02-17T21:27:20.739 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-115964116995 1-128849018883 2-141733920771' 2026-02-17T21:27:20.739 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:27:20.739 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T21:27:20.818 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789955 2026-02-17T21:27:20.819 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789955 2026-02-17T21:27:20.819 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-115964116995 1-128849018883 2-141733920771 3-158913789955' 2026-02-17T21:27:20.819 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:27:20.819 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-115964116995 2026-02-17T21:27:20.820 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:27:20.820 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:27:20.821 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-115964116995 2026-02-17T21:27:20.821 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:27:20.822 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964116995 2026-02-17T21:27:20.823 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 115964116995 2026-02-17T21:27:20.823 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 115964116995' 2026-02-17T21:27:20.823 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:27:21.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 115964116995 2026-02-17T21:27:21.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:27:22.017 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:27:22.018 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:27:22.221 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964116995 -lt 115964116995 2026-02-17T21:27:22.222 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:27:22.222 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-128849018883 2026-02-17T21:27:22.223 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:27:22.224 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:27:22.225 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-128849018883 2026-02-17T21:27:22.225 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:27:22.226 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018883 2026-02-17T21:27:22.226 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 128849018883' 2026-02-17T21:27:22.226 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 128849018883 2026-02-17T21:27:22.227 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:27:22.421 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018883 -lt 128849018883 2026-02-17T21:27:22.422 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:27:22.422 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-141733920771 2026-02-17T21:27:22.423 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:27:22.424 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:27:22.424 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-141733920771 2026-02-17T21:27:22.425 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:27:22.426 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=141733920771 2026-02-17T21:27:22.426 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 141733920771' 2026-02-17T21:27:22.426 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 141733920771 2026-02-17T21:27:22.426 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:27:22.628 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920771 -lt 141733920771 2026-02-17T21:27:22.629 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:27:22.629 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-158913789955 2026-02-17T21:27:22.629 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:27:22.630 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T21:27:22.631 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-158913789955 2026-02-17T21:27:22.631 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:27:22.633 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789955 2026-02-17T21:27:22.633 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.3 seq 158913789955 2026-02-17T21:27:22.633 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 158913789955' 2026-02-17T21:27:22.634 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T21:27:22.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789955 -lt 158913789955 2026-02-17T21:27:22.837 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:27:22.838 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:27:22.838 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:27:23.092 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T21:27:23.093 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:27:23.093 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:27:23.093 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:27:23.093 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:27:23.093 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:27:23.093 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:27:23.093 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:27:23.286 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T21:27:23.287 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:27:23.287 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:27:23.287 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:27:23.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T21:27:23.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T21:27:23.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T21:27:23.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:22: run: teardown td/ec-error-rollforward 2026-02-17T21:27:23.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/ec-error-rollforward 2026-02-17T21:27:23.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:27:23.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/ec-error-rollforward KILL 2026-02-17T21:27:23.544 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:27:23.544 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:27:23.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:27:23.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:27:23.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:27:23.855 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:27:23.855 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:27:23.857 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:27:23.857 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:27:23.859 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:27:23.859 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:27:23.859 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:27:23.860 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:27:23.860 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:27:23.861 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:27:23.861 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:27:23.862 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:27:23.863 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:27:23.863 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/ec-error-rollforward 2026-02-17T21:27:23.875 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:27:23.875 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:23.875 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:27:23.875 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.74709 2026-02-17T21:27:23.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:27:23.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:27:23.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T21:27:23.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/ec-error-rollforward 0 2026-02-17T21:27:23.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/ec-error-rollforward 2026-02-17T21:27:23.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T21:27:23.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/ec-error-rollforward KILL 2026-02-17T21:27:23.877 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:27:23.877 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:27:23.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:27:23.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:27:23.878 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:27:23.880 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:27:23.880 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:27:23.881 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:27:23.881 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:27:23.882 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:27:23.883 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:27:23.883 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:27:23.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:27:23.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:27:23.884 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:27:23.885 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:27:23.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:27:23.887 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T21:27:23.887 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/ec-error-rollforward 2026-02-17T21:27:23.888 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:27:23.888 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:23.888 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.74709 2026-02-17T21:27:23.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.74709 2026-02-17T21:27:23.889 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:27:23.889 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:27:23.889 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T21:27:23.890 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T21:27:23.890 DEBUG:teuthology.orchestra.run.trial031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T21:27:23.945 INFO:tasks.workunit:Running workunit osd/osd-bench.sh... 2026-02-17T21:27:23.945 DEBUG:teuthology.orchestra.run.trial031:workunit test osd/osd-bench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d855f53b89fdcec760fd9232a5fb55ed4fb111a1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh 2026-02-17T21:27:23.995 INFO:tasks.workunit.client.0.trial031.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-bench 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:22: run: local dir=td/osd-bench 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:23: run: shift 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:25: run: export CEPH_MON=127.0.0.1:7106 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:25: run: CEPH_MON=127.0.0.1:7106 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:26: run: export CEPH_ARGS 2026-02-17T21:27:23.998 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:27: run: uuidgen 2026-02-17T21:27:23.999 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:27: run: CEPH_ARGS+='--fsid=a3a8e0c0-3d65-4823-b15a-8f3ee32520cc --auth-supported=none ' 2026-02-17T21:27:23.999 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7106 ' 2026-02-17T21:27:23.999 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:29: run: CEPH_ARGS+='--debug-bluestore 20 ' 2026-02-17T21:27:23.999 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:31: run: set 2026-02-17T21:27:23.999 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T21:27:24.001 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:31: run: local funcs=TEST_bench 2026-02-17T21:27:24.001 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:32: run: for func in $funcs 2026-02-17T21:27:24.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:33: run: setup td/osd-bench 2026-02-17T21:27:24.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-bench 2026-02-17T21:27:24.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-bench 2026-02-17T21:27:24.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bench 2026-02-17T21:27:24.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:27:24.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bench KILL 2026-02-17T21:27:24.002 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:27:24.002 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:27:24.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:27:24.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:27:24.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:27:24.004 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:27:24.004 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:27:24.005 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:27:24.005 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:27:24.006 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:27:24.006 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:27:24.006 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:27:24.007 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:27:24.008 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:27:24.008 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:27:24.008 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:27:24.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:27:24.011 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:27:24.011 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bench 2026-02-17T21:27:24.012 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:27:24.012 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:24.012 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.83932 2026-02-17T21:27:24.013 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.83932 2026-02-17T21:27:24.013 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:27:24.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:27:24.014 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-bench 2026-02-17T21:27:24.015 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T21:27:24.015 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:24.015 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.83932 2026-02-17T21:27:24.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.83932 2026-02-17T21:27:24.017 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T21:27:24.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T21:27:24.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T21:27:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T21:27:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-bench 1' TERM HUP INT 2026-02-17T21:27:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:34: run: TEST_bench td/osd-bench 2026-02-17T21:27:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:40: TEST_bench: local dir=td/osd-bench 2026-02-17T21:27:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:42: TEST_bench: run_mon td/osd-bench a 2026-02-17T21:27:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-bench 2026-02-17T21:27:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T21:27:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T21:27:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T21:27:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-bench/a 2026-02-17T21:27:24.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bench/a --run-dir=td/osd-bench 2026-02-17T21:27:24.040 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T21:27:24.040 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:27:24.041 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:27:24.041 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:27:24.041 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:24.041 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.83932 2026-02-17T21:27:24.041 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.83932/$cluster-$name.asok' 2026-02-17T21:27:24.041 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bench/a '--log-file=td/osd-bench/$name.log' '--admin-socket=/tmp/ceph-asok.83932/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bench/log --run-dir=td/osd-bench '--pid-file=td/osd-bench/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T21:27:24.069 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T21:27:24.070 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T21:27:24.070 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:27:24.070 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:27:24.070 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T21:27:24.070 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T21:27:24.072 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:27:24.072 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:27:24.072 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:27:24.072 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:27:24.072 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:24.072 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.83932 2026-02-17T21:27:24.072 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.83932/ceph-mon.a.asok 2026-02-17T21:27:24.072 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:27:24.072 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.83932/ceph-mon.a.asok config get fsid 2026-02-17T21:27:24.145 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T21:27:24.145 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:27:24.145 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:27:24.146 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T21:27:24.146 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T21:27:24.147 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:27:24.147 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:27:24.147 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:27:24.147 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:27:24.147 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:24.147 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.83932 2026-02-17T21:27:24.147 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.83932/ceph-mon.a.asok 2026-02-17T21:27:24.147 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:27:24.148 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.83932/ceph-mon.a.asok config get mon_host 2026-02-17T21:27:24.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:43: TEST_bench: run_mgr td/osd-bench x 2026-02-17T21:27:24.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-bench 2026-02-17T21:27:24.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T21:27:24.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T21:27:24.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T21:27:24.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-bench/x 2026-02-17T21:27:24.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T21:27:24.313 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T21:27:24.313 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:27:24.313 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:27:24.313 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:27:24.313 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:24.313 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.83932 2026-02-17T21:27:24.313 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.83932/$cluster-$name.asok' 2026-02-17T21:27:24.314 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:27:24.314 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bench/x '--log-file=td/osd-bench/$name.log' '--admin-socket=/tmp/ceph-asok.83932/$cluster-$name.asok' --run-dir=td/osd-bench '--pid-file=td/osd-bench/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:27:24.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:44: TEST_bench: run_osd td/osd-bench 0 2026-02-17T21:27:24.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bench 2026-02-17T21:27:24.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:27:24.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T21:27:24.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:27:24.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bench/0 2026-02-17T21:27:24.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a3a8e0c0-3d65-4823-b15a-8f3ee32520cc --auth-supported=none --mon-host=127.0.0.1:7106 --debug-bluestore 20 ' 2026-02-17T21:27:24.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:27:24.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:27:24.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:27:24.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bench/0' 2026-02-17T21:27:24.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bench/0/journal' 2026-02-17T21:27:24.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:27:24.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:27:24.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bench' 2026-02-17T21:27:24.336 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:27:24.336 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:27:24.336 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:27:24.336 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:27:24.336 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:24.336 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.83932 2026-02-17T21:27:24.337 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.83932/$cluster-$name.asok' 2026-02-17T21:27:24.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.83932/$cluster-$name.asok' 2026-02-17T21:27:24.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:27:24.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:27:24.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:27:24.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bench/$name.log' 2026-02-17T21:27:24.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bench/$name.pid' 2026-02-17T21:27:24.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:27:24.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:27:24.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:27:24.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:27:24.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:27:24.337 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:27:24.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bench/0 2026-02-17T21:27:24.338 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:27:24.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9b294322-058a-45ad-b951-9e5788441129 2026-02-17T21:27:24.338 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 9b294322-058a-45ad-b951-9e5788441129' 2026-02-17T21:27:24.339 INFO:tasks.workunit.client.0.trial031.stdout:add osd0 9b294322-058a-45ad-b951-9e5788441129 2026-02-17T21:27:24.339 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:27:24.347 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA83ZRpX2nBFBAAnUbnkASx+aWFW9Wfuxi2Uw== 2026-02-17T21:27:24.347 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA83ZRpX2nBFBAAnUbnkASx+aWFW9Wfuxi2Uw=="}' 2026-02-17T21:27:24.347 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9b294322-058a-45ad-b951-9e5788441129 -i td/osd-bench/0/new.json 2026-02-17T21:27:24.457 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:27:24.465 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bench/0/new.json 2026-02-17T21:27:24.466 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a3a8e0c0-3d65-4823-b15a-8f3ee32520cc --auth-supported=none --mon-host=127.0.0.1:7106 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bench/0 --osd-journal=td/osd-bench/0/journal --chdir= --run-dir=td/osd-bench '--admin-socket=/tmp/ceph-asok.83932/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bench/$name.log' '--pid-file=td/osd-bench/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA83ZRpX2nBFBAAnUbnkASx+aWFW9Wfuxi2Uw== --osd-uuid 9b294322-058a-45ad-b951-9e5788441129 2026-02-17T21:27:24.480 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:24.478+0000 7fa1e3c038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:24.482 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:24.482+0000 7fa1e3c038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:24.483 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:24.482+0000 7fa1e3c038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:24.483 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:24.482+0000 7fa1e3c038c0 -1 bdev(0x555faf403c00 td/osd-bench/0/block) open stat got: (1) Operation not permitted 2026-02-17T21:27:24.484 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:24.482+0000 7fa1e3c038c0 -1 bluestore(td/osd-bench/0) _read_fsid unparsable uuid 2026-02-17T21:27:27.099 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bench/0/keyring 2026-02-17T21:27:27.100 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:27:27.100 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T21:27:27.101 INFO:tasks.workunit.client.0.trial031.stdout:adding osd0 key to auth repository 2026-02-17T21:27:27.101 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bench/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:27:27.367 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:27:27.367 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T21:27:27.367 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a3a8e0c0-3d65-4823-b15a-8f3ee32520cc --auth-supported=none --mon-host=127.0.0.1:7106 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bench/0 --osd-journal=td/osd-bench/0/journal --chdir= --run-dir=td/osd-bench '--admin-socket=/tmp/ceph-asok.83932/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bench/$name.log' '--pid-file=td/osd-bench/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:27:27.367 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:27:27.367 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:27:27.367 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:27:27.389 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:27.386+0000 7f2ff0fe18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:27.390 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:27.390+0000 7f2ff0fe18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:27.392 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:27.390+0000 7f2ff0fe18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:27.567 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T21:27:27.567 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:27:27.567 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:27:27.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:27:27.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:27:27.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:27.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:27:27.568 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:27:27.569 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:27.569 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:27:27.768 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:28.770 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:28.770 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:28.770 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:27:28.770 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:27:28.771 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:28.771 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:27:28.935 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:28.934+0000 7f2ff0fe18c0 -1 Falling back to public interface 2026-02-17T21:27:28.966 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:29.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:29.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:29.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:27:29.968 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:27:29.968 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:29.968 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:27:30.167 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:31.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:31.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:31.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:27:31.169 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:27:31.169 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:31.169 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:27:31.360 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:31.931 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:31.930+0000 7f2ff0fe18c0 -1 osd.0 0 log_to_monitors true 2026-02-17T21:27:32.361 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:32.361 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:32.361 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:27:32.362 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:27:32.362 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:32.362 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:27:32.560 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:33.562 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:33.562 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:33.562 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:27:33.563 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:27:33.563 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:33.563 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:27:33.756 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4164448040,v1:127.0.0.1:6803/4164448040] [v2:127.0.0.1:6804/4164448040,v1:127.0.0.1:6805/4164448040] exists,up 9b294322-058a-45ad-b951-9e5788441129 2026-02-17T21:27:33.756 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:27:33.756 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:27:33.756 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:27:33.757 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:47: TEST_bench: CEPH_ARGS= 2026-02-17T21:27:33.757 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:47: TEST_bench: ceph-conf --show-config-value osd_bench_small_size_max_iops 2026-02-17T21:27:33.765 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:47: TEST_bench: local osd_bench_small_size_max_iops=100 2026-02-17T21:27:33.765 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:49: TEST_bench: CEPH_ARGS= 2026-02-17T21:27:33.766 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:49: TEST_bench: ceph-conf --show-config-value osd_bench_large_size_max_throughput 2026-02-17T21:27:33.780 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:49: TEST_bench: local osd_bench_large_size_max_throughput=104857600 2026-02-17T21:27:33.780 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:51: TEST_bench: CEPH_ARGS= 2026-02-17T21:27:33.780 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:51: TEST_bench: ceph-conf --show-config-value osd_bench_max_block_size 2026-02-17T21:27:33.789 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:51: TEST_bench: local osd_bench_max_block_size=67108864 2026-02-17T21:27:33.789 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:53: TEST_bench: CEPH_ARGS= 2026-02-17T21:27:33.789 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:53: TEST_bench: ceph-conf --show-config-value osd_bench_duration 2026-02-17T21:27:33.803 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:53: TEST_bench: local osd_bench_duration=30 2026-02-17T21:27:33.803 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:58: TEST_bench: expect_failure td/osd-bench osd_bench_max_block_size ceph tell osd.0 bench 1024 67108865 2026-02-17T21:27:33.803 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: local dir=td/osd-bench 2026-02-17T21:27:33.804 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2026: expect_failure: shift 2026-02-17T21:27:33.804 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2027: expect_failure: local expected=osd_bench_max_block_size 2026-02-17T21:27:33.804 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: shift 2026-02-17T21:27:33.804 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: local success 2026-02-17T21:27:33.804 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2031: expect_failure: ceph tell osd.0 bench 1024 67108865 2026-02-17T21:27:33.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2034: expect_failure: success=false 2026-02-17T21:27:33.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: false 2026-02-17T21:27:33.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: grep --quiet osd_bench_max_block_size td/osd-bench/out 2026-02-17T21:27:33.876 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2041: expect_failure: return 0 2026-02-17T21:27:33.876 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:64: TEST_bench: local bsize=1024 2026-02-17T21:27:33.876 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:65: TEST_bench: local max_count=3072000 2026-02-17T21:27:33.876 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:66: TEST_bench: expect_failure td/osd-bench bench_small_size_max_iops ceph tell osd.0 bench 3072001 1024 2026-02-17T21:27:33.876 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: local dir=td/osd-bench 2026-02-17T21:27:33.876 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2026: expect_failure: shift 2026-02-17T21:27:33.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2027: expect_failure: local expected=bench_small_size_max_iops 2026-02-17T21:27:33.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: shift 2026-02-17T21:27:33.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: local success 2026-02-17T21:27:33.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2031: expect_failure: ceph tell osd.0 bench 3072001 1024 2026-02-17T21:27:33.947 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2034: expect_failure: success=false 2026-02-17T21:27:33.947 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: false 2026-02-17T21:27:33.947 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: grep --quiet bench_small_size_max_iops td/osd-bench/out 2026-02-17T21:27:33.948 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2041: expect_failure: return 0 2026-02-17T21:27:33.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:72: TEST_bench: local bsize=1048577 2026-02-17T21:27:33.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:73: TEST_bench: local max_count=3145728000 2026-02-17T21:27:33.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:74: TEST_bench: expect_failure td/osd-bench osd_bench_large_size_max_throughput ceph tell osd.0 bench 3145728001 1048577 2026-02-17T21:27:33.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: local dir=td/osd-bench 2026-02-17T21:27:33.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2026: expect_failure: shift 2026-02-17T21:27:33.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2027: expect_failure: local expected=osd_bench_large_size_max_throughput 2026-02-17T21:27:33.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: shift 2026-02-17T21:27:33.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: local success 2026-02-17T21:27:33.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2031: expect_failure: ceph tell osd.0 bench 3145728001 1048577 2026-02-17T21:27:34.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2034: expect_failure: success=false 2026-02-17T21:27:34.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: false 2026-02-17T21:27:34.020 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: grep --quiet osd_bench_large_size_max_throughput td/osd-bench/out 2026-02-17T21:27:34.021 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2041: expect_failure: return 0 2026-02-17T21:27:34.021 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:80: TEST_bench: ceph tell osd.0 bench 2026-02-17T21:27:35.758 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:27:35.758 INFO:tasks.workunit.client.0.trial031.stdout: "bytes_written": 1073741824, 2026-02-17T21:27:35.759 INFO:tasks.workunit.client.0.trial031.stdout: "blocksize": 4194304, 2026-02-17T21:27:35.759 INFO:tasks.workunit.client.0.trial031.stdout: "elapsed_sec": 0.74407349599999995, 2026-02-17T21:27:35.759 INFO:tasks.workunit.client.0.trial031.stdout: "bytes_per_sec": 1443058823.8557553, 2026-02-17T21:27:35.759 INFO:tasks.workunit.client.0.trial031.stdout: "iops": 344.05203434366115 2026-02-17T21:27:35.759 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:27:35.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:84: TEST_bench: ceph tell osd.0 bench 10 14456 4444 3 2026-02-17T21:27:35.842 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:27:35.842 INFO:tasks.workunit.client.0.trial031.stdout: "bytes_written": 10, 2026-02-17T21:27:35.842 INFO:tasks.workunit.client.0.trial031.stdout: "blocksize": 14456, 2026-02-17T21:27:35.842 INFO:tasks.workunit.client.0.trial031.stdout: "elapsed_sec": 0.000400185, 2026-02-17T21:27:35.842 INFO:tasks.workunit.client.0.trial031.stdout: "bytes_per_sec": 24988.442845184101, 2026-02-17T21:27:35.842 INFO:tasks.workunit.client.0.trial031.stdout: "iops": 1.7285862510503667 2026-02-17T21:27:35.842 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:27:35.849 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:90: TEST_bench: ceph tell osd.0 bench 1 14456 2026-02-17T21:27:35.912 INFO:tasks.workunit.client.0.trial031.stdout:{ 2026-02-17T21:27:35.912 INFO:tasks.workunit.client.0.trial031.stdout: "bytes_written": 1, 2026-02-17T21:27:35.912 INFO:tasks.workunit.client.0.trial031.stdout: "blocksize": 14456, 2026-02-17T21:27:35.913 INFO:tasks.workunit.client.0.trial031.stdout: "elapsed_sec": 0.00038112999999999998, 2026-02-17T21:27:35.913 INFO:tasks.workunit.client.0.trial031.stdout: "bytes_per_sec": 2623.7766641303492, 2026-02-17T21:27:35.913 INFO:tasks.workunit.client.0.trial031.stdout: "iops": 0.18150087604664839 2026-02-17T21:27:35.913 INFO:tasks.workunit.client.0.trial031.stdout:} 2026-02-17T21:27:35.920 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:35: run: teardown td/osd-bench 2026-02-17T21:27:35.920 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bench 2026-02-17T21:27:35.920 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:27:35.920 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bench KILL 2026-02-17T21:27:35.920 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:27:35.920 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:27:35.920 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:27:35.920 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:27:35.921 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:27:36.028 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:27:36.028 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:27:36.029 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:27:36.029 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:27:36.031 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:27:36.031 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:27:36.031 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:27:36.032 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:27:36.032 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:27:36.033 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:27:36.033 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:27:36.034 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:27:36.034 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:27:36.035 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bench 2026-02-17T21:27:36.065 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:27:36.065 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:36.065 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.83932 2026-02-17T21:27:36.066 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.83932 2026-02-17T21:27:36.067 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:27:36.067 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:27:36.067 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T21:27:36.068 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-bench 0 2026-02-17T21:27:36.068 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bench 2026-02-17T21:27:36.068 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T21:27:36.068 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bench KILL 2026-02-17T21:27:36.068 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:27:36.068 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:27:36.068 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:27:36.068 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:27:36.068 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:27:36.069 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:27:36.070 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:27:36.071 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:27:36.071 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:27:36.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:27:36.072 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:27:36.072 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:27:36.073 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:27:36.073 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:27:36.073 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:27:36.074 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:27:36.075 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:27:36.076 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T21:27:36.076 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bench 2026-02-17T21:27:36.077 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:27:36.077 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:36.078 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.83932 2026-02-17T21:27:36.078 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.83932 2026-02-17T21:27:36.079 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:27:36.079 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:27:36.079 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T21:27:36.080 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T21:27:36.080 DEBUG:teuthology.orchestra.run.trial031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T21:27:36.130 INFO:tasks.workunit:Running workunit osd/osd-bluefs-volume-ops.sh... 2026-02-17T21:27:36.130 DEBUG:teuthology.orchestra.run.trial031:workunit test osd/osd-bluefs-volume-ops.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d855f53b89fdcec760fd9232a5fb55ed4fb111a1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh 2026-02-17T21:27:36.183 INFO:tasks.workunit.client.0.trial031.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T21:27:36.188 INFO:tasks.workunit.client.0.trial031.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T21:27:36.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T21:27:36.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T21:27:36.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T21:27:36.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T21:27:36.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T21:27:36.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T21:27:36.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T21:27:36.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T21:27:36.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-bluefs-volume-ops 2026-02-17T21:27:36.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:8: run: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:27:36.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:9: run: shift 2026-02-17T21:27:36.189 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:11: run: set 2026-02-17T21:27:36.190 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:11: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T21:27:36.192 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:11: run: local 'funcs=TEST_bluestore 2026-02-17T21:27:36.192 INFO:tasks.workunit.client.0.trial031.stderr:TEST_bluestore2 2026-02-17T21:27:36.192 INFO:tasks.workunit.client.0.trial031.stderr:TEST_bluestore_expand' 2026-02-17T21:27:36.192 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:12: run: for func in $funcs 2026-02-17T21:27:36.192 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:13: run: setup td/osd-bluefs-volume-ops 2026-02-17T21:27:36.192 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:27:36.192 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-bluefs-volume-ops 2026-02-17T21:27:36.192 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:27:36.192 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:27:36.192 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-17T21:27:36.193 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:27:36.193 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:27:36.193 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:27:36.193 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:27:36.193 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:27:36.194 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:27:36.194 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:27:36.195 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:27:36.195 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:27:36.196 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:27:36.197 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:27:36.197 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:27:36.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:27:36.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:27:36.199 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:27:36.199 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:27:36.199 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:27:36.200 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:27:36.200 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-17T21:27:36.202 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:27:36.202 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:36.202 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:27:36.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86056 2026-02-17T21:27:36.203 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:27:36.203 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:27:36.203 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-bluefs-volume-ops 2026-02-17T21:27:36.204 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T21:27:36.205 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:36.205 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:27:36.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.86056 2026-02-17T21:27:36.206 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T21:27:36.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T21:27:36.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T21:27:36.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T21:27:36.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-bluefs-volume-ops 1' TERM HUP INT 2026-02-17T21:27:36.208 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: TEST_bluestore td/osd-bluefs-volume-ops 2026-02-17T21:27:36.208 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:20: TEST_bluestore: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:27:36.208 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:22: TEST_bluestore: ulimit -n 2026-02-17T21:27:36.209 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:22: TEST_bluestore: local flimit=4096 2026-02-17T21:27:36.209 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:23: TEST_bluestore: '[' 4096 -lt 1536 ']' 2026-02-17T21:27:36.209 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:26: TEST_bluestore: export CEPH_MON=127.0.0.1:7146 2026-02-17T21:27:36.209 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:26: TEST_bluestore: CEPH_MON=127.0.0.1:7146 2026-02-17T21:27:36.209 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:27: TEST_bluestore: export CEPH_ARGS 2026-02-17T21:27:36.209 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:28: TEST_bluestore: uuidgen 2026-02-17T21:27:36.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:28: TEST_bluestore: CEPH_ARGS+='--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none ' 2026-02-17T21:27:36.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:29: TEST_bluestore: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-17T21:27:36.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:30: TEST_bluestore: CEPH_ARGS+='--bluestore_block_size=2147483648 ' 2026-02-17T21:27:36.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:31: TEST_bluestore: CEPH_ARGS+='--bluestore_block_db_create=true ' 2026-02-17T21:27:36.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:32: TEST_bluestore: CEPH_ARGS+='--bluestore_block_db_size=1073741824 ' 2026-02-17T21:27:36.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:33: TEST_bluestore: CEPH_ARGS+='--bluestore_block_wal_size=536870912 ' 2026-02-17T21:27:36.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:34: TEST_bluestore: CEPH_ARGS+='--bluestore_block_wal_create=true ' 2026-02-17T21:27:36.212 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:35: TEST_bluestore: CEPH_ARGS+='--bluestore_fsck_on_mount=true ' 2026-02-17T21:27:36.212 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:37: TEST_bluestore: CEPH_ARGS+='--bluestore_allocation_from_file=1 ' 2026-02-17T21:27:36.212 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:39: TEST_bluestore: run_mon td/osd-bluefs-volume-ops a 2026-02-17T21:27:36.212 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:27:36.212 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T21:27:36.212 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T21:27:36.212 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T21:27:36.212 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-bluefs-volume-ops/a 2026-02-17T21:27:36.212 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bluefs-volume-ops/a --run-dir=td/osd-bluefs-volume-ops 2026-02-17T21:27:36.233 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T21:27:36.233 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:27:36.233 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:27:36.233 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:27:36.234 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:36.234 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:27:36.234 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:27:36.234 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bluefs-volume-ops/a '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bluefs-volume-ops/log --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T21:27:36.261 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T21:27:36.262 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T21:27:36.262 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:27:36.262 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:27:36.263 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T21:27:36.264 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T21:27:36.264 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:27:36.264 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:27:36.264 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:27:36.264 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:27:36.264 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:36.264 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:27:36.265 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86056/ceph-mon.a.asok 2026-02-17T21:27:36.265 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:27:36.265 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.86056/ceph-mon.a.asok config get fsid 2026-02-17T21:27:36.328 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T21:27:36.328 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:27:36.329 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:27:36.329 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T21:27:36.329 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T21:27:36.330 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:27:36.330 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:27:36.330 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:27:36.330 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:27:36.330 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:36.330 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:27:36.330 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86056/ceph-mon.a.asok 2026-02-17T21:27:36.331 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:27:36.331 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.86056/ceph-mon.a.asok config get mon_host 2026-02-17T21:27:36.386 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:40: TEST_bluestore: run_mgr td/osd-bluefs-volume-ops x 2026-02-17T21:27:36.386 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:27:36.386 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T21:27:36.386 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T21:27:36.386 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T21:27:36.386 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-bluefs-volume-ops/x 2026-02-17T21:27:36.386 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T21:27:36.493 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T21:27:36.493 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:27:36.493 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:27:36.494 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:27:36.494 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:36.494 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:27:36.494 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:27:36.494 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:27:36.495 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bluefs-volume-ops/x '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:27:36.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:41: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 0 2026-02-17T21:27:36.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:27:36.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:27:36.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T21:27:36.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:27:36.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-17T21:27:36.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:27:36.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:27:36.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:27:36.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:27:36.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-17T21:27:36.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-17T21:27:36.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:27:36.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:27:36.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:27:36.516 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:27:36.516 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:27:36.516 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:27:36.516 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:27:36.516 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:36.516 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:27:36.516 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:27:36.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:27:36.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:27:36.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:27:36.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:27:36.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:27:36.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:27:36.517 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:27:36.517 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:27:36.517 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:27:36.517 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:27:36.517 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:27:36.517 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:27:36.517 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-17T21:27:36.518 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:27:36.518 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=df635ca7-b5c9-471f-8ba7-205a3bf37cdb 2026-02-17T21:27:36.518 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 df635ca7-b5c9-471f-8ba7-205a3bf37cdb' 2026-02-17T21:27:36.518 INFO:tasks.workunit.client.0.trial031.stdout:add osd0 df635ca7-b5c9-471f-8ba7-205a3bf37cdb 2026-02-17T21:27:36.519 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:27:36.527 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBI3ZRp6It5HxAAxNqpacpRybVtsW69/bBL9A== 2026-02-17T21:27:36.527 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBI3ZRp6It5HxAAxNqpacpRybVtsW69/bBL9A=="}' 2026-02-17T21:27:36.527 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new df635ca7-b5c9-471f-8ba7-205a3bf37cdb -i td/osd-bluefs-volume-ops/0/new.json 2026-02-17T21:27:36.637 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:27:36.646 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/0/new.json 2026-02-17T21:27:36.647 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBI3ZRp6It5HxAAxNqpacpRybVtsW69/bBL9A== --osd-uuid df635ca7-b5c9-471f-8ba7-205a3bf37cdb 2026-02-17T21:27:36.663 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:36.662+0000 7f04659798c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:36.664 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:36.662+0000 7f04659798c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:36.665 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:36.662+0000 7f04659798c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:36.666 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:36.662+0000 7f04659798c0 -1 bdev(0x55a8f2ae1c00 td/osd-bluefs-volume-ops/0/block) open stat got: (1) Operation not permitted 2026-02-17T21:27:36.666 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:36.662+0000 7f04659798c0 -1 bluestore(td/osd-bluefs-volume-ops/0) _read_fsid unparsable uuid 2026-02-17T21:27:43.558 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/0/keyring 2026-02-17T21:27:43.559 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:27:43.560 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T21:27:43.560 INFO:tasks.workunit.client.0.trial031.stdout:adding osd0 key to auth repository 2026-02-17T21:27:43.560 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:27:43.830 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:27:43.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T21:27:43.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:27:43.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:27:43.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:27:43.831 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:27:43.845 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:43.841+0000 7f7fea6948c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:43.846 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:43.845+0000 7f7fea6948c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:43.848 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:43.845+0000 7f7fea6948c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:44.029 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T21:27:44.029 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:27:44.030 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:27:44.030 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:27:44.030 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:27:44.030 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:44.030 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:27:44.030 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:27:44.031 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:44.031 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:27:44.224 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:45.226 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:45.226 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:45.226 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:27:45.227 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:27:45.227 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:45.227 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:27:45.424 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:45.875 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:45.873+0000 7f7fea6948c0 -1 Falling back to public interface 2026-02-17T21:27:46.425 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:46.425 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:46.425 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:27:46.425 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:27:46.425 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:46.425 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:27:46.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:47.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:47.621 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:47.621 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:27:47.621 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:27:47.621 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:47.621 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:27:47.814 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:48.815 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:48.815 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:48.816 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:27:48.816 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:27:48.816 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:48.816 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:27:49.008 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:50.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:50.009 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:50.010 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:27:50.010 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:27:50.010 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:50.011 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:27:50.212 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:51.198 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:51.197+0000 7f7fea6948c0 -1 osd.0 0 log_to_monitors true 2026-02-17T21:27:51.213 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:51.213 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:51.213 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:27:51.214 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:27:51.214 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:51.214 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:27:51.408 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:27:52.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:27:52.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:27:52.409 INFO:tasks.workunit.client.0.trial031.stdout:7 2026-02-17T21:27:52.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-17T21:27:52.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:27:52.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:27:52.602 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3984650747,v1:127.0.0.1:6803/3984650747] [v2:127.0.0.1:6804/3984650747,v1:127.0.0.1:6805/3984650747] exists,up df635ca7-b5c9-471f-8ba7-205a3bf37cdb 2026-02-17T21:27:52.603 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:27:52.603 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:27:52.603 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:27:52.603 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:42: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-17T21:27:52.606 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:42: TEST_bluestore: osd_pid0=87534 2026-02-17T21:27:52.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:43: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 1 2026-02-17T21:27:52.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:27:52.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:27:52.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T21:27:52.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:27:52.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-17T21:27:52.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:27:52.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:27:52.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:27:52.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:27:52.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-17T21:27:52.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-17T21:27:52.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:27:52.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:27:52.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:27:52.608 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:27:52.608 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:27:52.608 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:27:52.608 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:27:52.608 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:27:52.608 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:27:52.608 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:27:52.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:27:52.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:27:52.609 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:27:52.609 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:27:52.609 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:27:52.609 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:27:52.609 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:27:52.609 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:27:52.609 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:27:52.609 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:27:52.609 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:27:52.609 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:27:52.609 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-17T21:27:52.610 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:27:52.610 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3060462c-e701-42c9-891d-76dd092c903e 2026-02-17T21:27:52.610 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 3060462c-e701-42c9-891d-76dd092c903e' 2026-02-17T21:27:52.610 INFO:tasks.workunit.client.0.trial031.stdout:add osd1 3060462c-e701-42c9-891d-76dd092c903e 2026-02-17T21:27:52.611 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:27:52.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBY3ZRpZJADJRAAwB7tsTb+JgwT6McsNNy+Kg== 2026-02-17T21:27:52.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBY3ZRpZJADJRAAwB7tsTb+JgwT6McsNNy+Kg=="}' 2026-02-17T21:27:52.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3060462c-e701-42c9-891d-76dd092c903e -i td/osd-bluefs-volume-ops/1/new.json 2026-02-17T21:27:52.830 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:27:52.839 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/1/new.json 2026-02-17T21:27:52.840 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBY3ZRpZJADJRAAwB7tsTb+JgwT6McsNNy+Kg== --osd-uuid 3060462c-e701-42c9-891d-76dd092c903e 2026-02-17T21:27:52.855 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:52.853+0000 7ff06ed8a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:52.856 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:52.853+0000 7ff06ed8a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:52.857 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:52.853+0000 7ff06ed8a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:52.858 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:52.857+0000 7ff06ed8a8c0 -1 bdev(0x5594f7869c00 td/osd-bluefs-volume-ops/1/block) open stat got: (1) Operation not permitted 2026-02-17T21:27:52.858 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:52.857+0000 7ff06ed8a8c0 -1 bluestore(td/osd-bluefs-volume-ops/1) _read_fsid unparsable uuid 2026-02-17T21:27:59.678 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/1/keyring 2026-02-17T21:27:59.678 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:27:59.679 INFO:tasks.workunit.client.0.trial031.stdout:adding osd1 key to auth repository 2026-02-17T21:27:59.680 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T21:27:59.680 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:27:59.956 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:27:59.956 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T21:27:59.956 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:27:59.957 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:27:59.957 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:27:59.957 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:27:59.970 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:59.969+0000 7f92c59c98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:59.972 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:59.969+0000 7f92c59c98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:27:59.973 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:27:59.969+0000 7f92c59c98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:28:00.151 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T21:28:00.152 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:28:00.152 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:28:00.152 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:28:00.152 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:28:00.152 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:00.152 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:28:00.152 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:28:00.152 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:00.152 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:28:00.353 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:01.354 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:01.354 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:01.354 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:28:01.354 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:28:01.354 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:01.355 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:28:01.553 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:01.968 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:01.965+0000 7f92c59c98c0 -1 Falling back to public interface 2026-02-17T21:28:02.554 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:02.554 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:02.555 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:28:02.555 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:28:02.555 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:02.555 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:28:02.756 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:03.758 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:03.758 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:03.758 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:28:03.759 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:28:03.759 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:03.759 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:28:03.961 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:04.962 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:04.962 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:04.962 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:28:04.962 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:28:04.962 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:04.962 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:28:05.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:06.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:06.166 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:28:06.167 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:06.167 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:28:06.167 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:06.167 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:28:06.369 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:07.370 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:07.370 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:07.371 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:28:07.372 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:28:07.372 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:07.372 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:28:07.503 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:07.501+0000 7f92c59c98c0 -1 osd.1 0 log_to_monitors true 2026-02-17T21:28:07.571 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:08.572 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:08.573 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:08.573 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-17T21:28:08.573 INFO:tasks.workunit.client.0.trial031.stdout:7 2026-02-17T21:28:08.574 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:08.574 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:28:08.776 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/715771972,v1:127.0.0.1:6811/715771972] [v2:127.0.0.1:6812/715771972,v1:127.0.0.1:6813/715771972] exists,up 3060462c-e701-42c9-891d-76dd092c903e 2026-02-17T21:28:08.777 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:28:08.777 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:28:08.777 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:28:08.777 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:44: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-17T21:28:08.779 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:44: TEST_bluestore: osd_pid1=89808 2026-02-17T21:28:08.780 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:45: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 2 2026-02-17T21:28:08.780 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:28:08.780 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:28:08.780 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T21:28:08.780 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:28:08.780 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-17T21:28:08.780 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:28:08.780 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:28:08.780 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:28:08.780 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:28:08.781 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-17T21:28:08.781 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-17T21:28:08.781 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:28:08.781 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:28:08.781 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:28:08.781 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:28:08.781 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:28:08.781 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:28:08.781 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:28:08.781 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:28:08.781 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:28:08.783 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:28:08.783 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:28:08.783 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:28:08.783 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:28:08.783 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:28:08.783 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:28:08.783 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:28:08.784 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:28:08.784 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:28:08.784 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:28:08.784 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:28:08.784 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:28:08.784 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:28:08.784 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-17T21:28:08.784 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:28:08.784 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3d0ce96d-e4ec-4623-a2b6-711f320be599 2026-02-17T21:28:08.784 INFO:tasks.workunit.client.0.trial031.stdout:add osd2 3d0ce96d-e4ec-4623-a2b6-711f320be599 2026-02-17T21:28:08.785 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 3d0ce96d-e4ec-4623-a2b6-711f320be599' 2026-02-17T21:28:08.785 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:28:08.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBo3ZRpbmp7LxAAeDHM7hmZJzfMw8n7zWh3bw== 2026-02-17T21:28:08.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBo3ZRpbmp7LxAAeDHM7hmZJzfMw8n7zWh3bw=="}' 2026-02-17T21:28:08.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3d0ce96d-e4ec-4623-a2b6-711f320be599 -i td/osd-bluefs-volume-ops/2/new.json 2026-02-17T21:28:08.996 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:28:09.006 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/2/new.json 2026-02-17T21:28:09.007 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBo3ZRpbmp7LxAAeDHM7hmZJzfMw8n7zWh3bw== --osd-uuid 3d0ce96d-e4ec-4623-a2b6-711f320be599 2026-02-17T21:28:09.030 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:09.029+0000 7fbca4f988c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:28:09.032 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:09.029+0000 7fbca4f988c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:28:09.033 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:09.029+0000 7fbca4f988c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:28:09.034 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:09.033+0000 7fbca4f988c0 -1 bdev(0x557ff5a39c00 td/osd-bluefs-volume-ops/2/block) open stat got: (1) Operation not permitted 2026-02-17T21:28:09.034 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:09.033+0000 7fbca4f988c0 -1 bluestore(td/osd-bluefs-volume-ops/2) _read_fsid unparsable uuid 2026-02-17T21:28:15.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/2/keyring 2026-02-17T21:28:15.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:28:15.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T21:28:15.609 INFO:tasks.workunit.client.0.trial031.stdout:adding osd2 key to auth repository 2026-02-17T21:28:15.609 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:28:15.876 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:28:15.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T21:28:15.878 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:28:15.878 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:28:15.878 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:28:15.878 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:28:15.899 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:15.897+0000 7f4164a8a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:28:15.901 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:15.897+0000 7f4164a8a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:28:15.902 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:15.901+0000 7f4164a8a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:28:16.078 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T21:28:16.078 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:28:16.078 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:28:16.078 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:28:16.079 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:28:16.079 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:16.079 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:28:16.080 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:28:16.080 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:16.080 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:28:16.278 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:17.279 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:17.280 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:17.280 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:28:17.281 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:28:17.281 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:17.281 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:28:17.475 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:17.859 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:17.856+0000 7f4164a8a8c0 -1 Falling back to public interface 2026-02-17T21:28:18.477 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:18.477 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:18.477 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:28:18.477 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:28:18.478 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:18.478 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:28:18.673 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:19.674 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:19.675 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:19.675 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:28:19.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:28:19.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:19.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:28:19.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:20.878 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:20.879 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:20.879 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:28:20.879 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:28:20.880 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:20.880 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:28:21.073 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:22.074 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:22.074 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:22.074 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:28:22.074 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:28:22.074 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:22.074 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:28:22.269 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:23.186 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:23.184+0000 7f4164a8a8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T21:28:23.271 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:23.271 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:23.272 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:28:23.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:28:23.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:23.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:28:23.469 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:24.470 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:24.471 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:24.471 INFO:tasks.workunit.client.0.trial031.stdout:7 2026-02-17T21:28:24.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-17T21:28:24.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:24.472 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:28:24.673 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1290985965,v1:127.0.0.1:6819/1290985965] [v2:127.0.0.1:6820/1290985965,v1:127.0.0.1:6821/1290985965] exists,up 3d0ce96d-e4ec-4623-a2b6-711f320be599 2026-02-17T21:28:24.673 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:28:24.673 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:28:24.673 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:28:24.674 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:46: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-17T21:28:24.675 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:46: TEST_bluestore: osd_pid2=92661 2026-02-17T21:28:24.675 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:47: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 3 2026-02-17T21:28:24.675 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:28:24.675 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:28:24.675 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:28:24.676 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:28:24.677 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:28:24.678 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:28:24.678 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:28:24.678 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:28:24.678 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:28:24.678 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:28:24.678 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:28:24.678 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:28:24.679 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:28:24.679 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:28:24.679 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:28:24.679 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:28:24.679 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:28:24.679 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-17T21:28:24.680 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:28:24.680 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3c7e5245-677b-4faf-ac5c-593720b47b50 2026-02-17T21:28:24.680 INFO:tasks.workunit.client.0.trial031.stdout:add osd3 3c7e5245-677b-4faf-ac5c-593720b47b50 2026-02-17T21:28:24.680 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 3c7e5245-677b-4faf-ac5c-593720b47b50' 2026-02-17T21:28:24.680 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:28:24.691 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB43ZRpSAtKKRAAZj6Vjj44dkZ3NuOzNF8TNg== 2026-02-17T21:28:24.691 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB43ZRpSAtKKRAAZj6Vjj44dkZ3NuOzNF8TNg=="}' 2026-02-17T21:28:24.691 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3c7e5245-677b-4faf-ac5c-593720b47b50 -i td/osd-bluefs-volume-ops/3/new.json 2026-02-17T21:28:24.898 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:28:24.907 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/3/new.json 2026-02-17T21:28:24.908 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB43ZRpSAtKKRAAZj6Vjj44dkZ3NuOzNF8TNg== --osd-uuid 3c7e5245-677b-4faf-ac5c-593720b47b50 2026-02-17T21:28:24.923 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:24.920+0000 7f0d2effb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:28:24.924 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:24.920+0000 7f0d2effb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:28:24.925 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:24.924+0000 7f0d2effb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:28:24.926 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:24.924+0000 7f0d2effb8c0 -1 bdev(0x559db99dfc00 td/osd-bluefs-volume-ops/3/block) open stat got: (1) Operation not permitted 2026-02-17T21:28:24.926 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:24.924+0000 7f0d2effb8c0 -1 bluestore(td/osd-bluefs-volume-ops/3) _read_fsid unparsable uuid 2026-02-17T21:28:31.506 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/3/keyring 2026-02-17T21:28:31.506 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:28:31.507 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T21:28:31.508 INFO:tasks.workunit.client.0.trial031.stdout:adding osd3 key to auth repository 2026-02-17T21:28:31.508 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:28:31.777 INFO:tasks.workunit.client.0.trial031.stdout:start osd.3 2026-02-17T21:28:31.777 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T21:28:31.778 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:28:31.778 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:28:31.778 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:28:31.778 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:28:31.793 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:31.792+0000 7fd35cbb88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:28:31.795 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:31.792+0000 7fd35cbb88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:28:31.796 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:31.792+0000 7fd35cbb88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:28:31.971 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T21:28:31.971 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:28:31.971 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T21:28:31.971 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:28:31.971 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:28:31.972 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:31.972 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:28:31.972 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:28:31.972 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:31.973 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:28:32.178 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:33.180 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:33.180 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:33.180 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:28:33.180 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:28:33.181 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:33.181 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:28:33.382 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:33.779 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:33.776+0000 7fd35cbb88c0 -1 Falling back to public interface 2026-02-17T21:28:34.383 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:34.384 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:34.384 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:28:34.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:28:34.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:34.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:28:34.586 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:35.587 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:35.587 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:35.587 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:28:35.587 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:28:35.587 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:35.588 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:28:35.789 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:36.790 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:36.790 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:36.790 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:28:36.791 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:28:36.791 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:36.791 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:28:36.993 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:37.994 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:37.995 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:37.995 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:28:37.995 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:28:37.996 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:37.996 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:28:38.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:39.199 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:39.200 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:39.200 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:28:39.200 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:28:39.201 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:39.201 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:28:39.356 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:28:39.352+0000 7fd35cbb88c0 -1 osd.3 0 log_to_monitors true 2026-02-17T21:28:39.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:28:40.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:28:40.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:28:40.407 INFO:tasks.workunit.client.0.trial031.stdout:7 2026-02-17T21:28:40.407 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-17T21:28:40.407 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:28:40.407 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:28:40.605 INFO:tasks.workunit.client.0.trial031.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1113799551,v1:127.0.0.1:6827/1113799551] [v2:127.0.0.1:6828/1113799551,v1:127.0.0.1:6829/1113799551] exists,up 3c7e5245-677b-4faf-ac5c-593720b47b50 2026-02-17T21:28:40.605 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:28:40.605 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:28:40.605 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:28:40.606 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:48: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-17T21:28:40.606 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:48: TEST_bluestore: osd_pid3=95037 2026-02-17T21:28:40.607 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:50: TEST_bluestore: sleep 5 2026-02-17T21:28:45.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:52: TEST_bluestore: create_pool foo 16 2026-02-17T21:28:45.608 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 16 2026-02-17T21:28:45.857 INFO:tasks.workunit.client.0.trial031.stderr:pool 'foo' already exists 2026-02-17T21:28:45.867 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T21:28:46.869 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:55: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-17T21:28:46.889 INFO:tasks.workunit.client.0.trial031.stdout:hints = 1 2026-02-17T21:28:46.889 INFO:tasks.workunit.client.0.trial031.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-17T21:28:46.889 INFO:tasks.workunit.client.0.trial031.stdout:Object prefix: benchmark_data_trial031_96858 2026-02-17T21:28:46.889 INFO:tasks.workunit.client.0.trial031.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-17T21:28:46.889 INFO:tasks.workunit.client.0.trial031.stdout: 0 0 0 0 0 0 - 0 2026-02-17T21:28:47.889 INFO:tasks.workunit.client.0.trial031.stdout: 1 16 9432 9416 36.7782 36.7812 0.00174036 0.00169647 2026-02-17T21:28:48.889 INFO:tasks.workunit.client.0.trial031.stdout: 2 16 18876 18860 36.833 36.8906 0.00145267 0.00169475 2026-02-17T21:28:49.889 INFO:tasks.workunit.client.0.trial031.stdout: 3 16 28161 28145 36.6443 36.2695 0.00172799 0.00170379 2026-02-17T21:28:50.889 INFO:tasks.workunit.client.0.trial031.stdout: 4 16 37560 37544 36.6612 36.7148 0.00137093 0.00170314 2026-02-17T21:28:51.889 INFO:tasks.workunit.client.0.trial031.stdout: 5 16 47045 47029 36.7386 37.0508 0.00156696 0.00169952 2026-02-17T21:28:52.889 INFO:tasks.workunit.client.0.trial031.stdout: 6 16 56900 56884 37.0311 38.4961 0.00200837 0.00168614 2026-02-17T21:28:53.889 INFO:tasks.workunit.client.0.trial031.stdout: 7 16 66650 66634 37.1814 38.0859 0.00145926 0.00167939 2026-02-17T21:28:54.889 INFO:tasks.workunit.client.0.trial031.stdout: 8 16 76487 76471 37.3365 38.4258 0.00164551 0.00167244 2026-02-17T21:28:55.889 INFO:tasks.workunit.client.0.trial031.stdout: 9 16 87014 86998 37.7566 41.1211 0.00158575 0.00165384 2026-02-17T21:28:56.889 INFO:tasks.workunit.client.0.trial031.stdout: 10 16 96865 96849 37.8287 38.4805 0.00313365 0.00165054 2026-02-17T21:28:57.890 INFO:tasks.workunit.client.0.trial031.stdout: 11 16 107347 107331 38.1117 40.9453 0.00126911 0.00163839 2026-02-17T21:28:58.890 INFO:tasks.workunit.client.0.trial031.stdout: 12 16 117833 117817 38.3489 40.9609 0.00153924 0.00162829 2026-02-17T21:28:59.890 INFO:tasks.workunit.client.0.trial031.stdout: 13 16 128030 128014 38.4628 39.832 0.00476139 0.00162334 2026-02-17T21:29:00.890 INFO:tasks.workunit.client.0.trial031.stdout: 14 16 138395 138379 38.6072 40.4883 0.00110107 0.00161724 2026-02-17T21:29:01.890 INFO:tasks.workunit.client.0.trial031.stdout: 15 16 148053 148037 38.5483 37.7266 0.00141112 0.00161985 2026-02-17T21:29:02.890 INFO:tasks.workunit.client.0.trial031.stdout: 16 15 156992 156977 38.3215 34.9219 0.00164327 0.00162947 2026-02-17T21:29:03.890 INFO:tasks.workunit.client.0.trial031.stdout: 17 16 165934 165918 38.1216 34.9258 0.00141527 0.00163802 2026-02-17T21:29:04.890 INFO:tasks.workunit.client.0.trial031.stdout: 18 15 175009 174994 37.9732 35.4531 0.00221066 0.00164443 2026-02-17T21:29:05.890 INFO:tasks.workunit.client.0.trial031.stdout: 19 16 183726 183710 37.7664 34.0469 0.0014415 0.00165341 2026-02-17T21:29:06.890 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:29:06.893107+0000 min lat: 0.00063351 max lat: 0.011904 avg lat: 0.00165705 2026-02-17T21:29:06.891 INFO:tasks.workunit.client.0.trial031.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-17T21:29:06.891 INFO:tasks.workunit.client.0.trial031.stdout: 20 16 192976 192960 37.6846 36.1328 0.00213483 0.00165705 2026-02-17T21:29:07.890 INFO:tasks.workunit.client.0.trial031.stdout: 21 16 199154 199138 37.0392 24.1328 0.064775 0.00168422 2026-02-17T21:29:08.890 INFO:tasks.workunit.client.0.trial031.stdout: 22 16 199253 199237 35.3731 0.386719 0.121066 0.00175731 2026-02-17T21:29:09.890 INFO:tasks.workunit.client.0.trial031.stdout: 23 16 199358 199342 33.853 0.410156 0.435528 0.00184196 2026-02-17T21:29:10.890 INFO:tasks.workunit.client.0.trial031.stdout: 24 16 199471 199455 32.4609 0.441406 0.43478 0.00189615 2026-02-17T21:29:11.890 INFO:tasks.workunit.client.0.trial031.stdout: 25 16 199618 199602 31.1854 0.574219 0.0320992 0.00200071 2026-02-17T21:29:12.891 INFO:tasks.workunit.client.0.trial031.stdout: 26 16 199670 199654 29.9938 0.203125 0.257115 0.00205434 2026-02-17T21:29:13.891 INFO:tasks.workunit.client.0.trial031.stdout: 27 16 199699 199683 28.8871 0.113281 0.729535 0.00215401 2026-02-17T21:29:14.891 INFO:tasks.workunit.client.0.trial031.stdout: 28 16 204278 204262 28.4942 17.8867 0.00178965 0.00219197 2026-02-17T21:29:15.891 INFO:tasks.workunit.client.0.trial031.stdout: 29 16 213142 213126 28.7055 34.625 0.00141944 0.00217581 2026-02-17T21:29:16.891 INFO:tasks.workunit.client.0.trial031.stdout:Total time run: 30.0016 2026-02-17T21:29:16.891 INFO:tasks.workunit.client.0.trial031.stdout:Total writes made: 221557 2026-02-17T21:29:16.891 INFO:tasks.workunit.client.0.trial031.stdout:Write size: 4096 2026-02-17T21:29:16.891 INFO:tasks.workunit.client.0.trial031.stdout:Object size: 4096 2026-02-17T21:29:16.891 INFO:tasks.workunit.client.0.trial031.stdout:Bandwidth (MB/sec): 28.847 2026-02-17T21:29:16.892 INFO:tasks.workunit.client.0.trial031.stdout:Stddev Bandwidth: 15.466 2026-02-17T21:29:16.892 INFO:tasks.workunit.client.0.trial031.stdout:Max bandwidth (MB/sec): 41.1211 2026-02-17T21:29:16.892 INFO:tasks.workunit.client.0.trial031.stdout:Min bandwidth (MB/sec): 0.113281 2026-02-17T21:29:16.892 INFO:tasks.workunit.client.0.trial031.stdout:Average IOPS: 7384 2026-02-17T21:29:16.892 INFO:tasks.workunit.client.0.trial031.stdout:Stddev IOPS: 3959.3 2026-02-17T21:29:16.892 INFO:tasks.workunit.client.0.trial031.stdout:Max IOPS: 10527 2026-02-17T21:29:16.892 INFO:tasks.workunit.client.0.trial031.stdout:Min IOPS: 29 2026-02-17T21:29:16.892 INFO:tasks.workunit.client.0.trial031.stdout:Average Latency(s): 0.00216517 2026-02-17T21:29:16.892 INFO:tasks.workunit.client.0.trial031.stdout:Stddev Latency(s): 0.012822 2026-02-17T21:29:16.892 INFO:tasks.workunit.client.0.trial031.stdout:Max latency(s): 0.734914 2026-02-17T21:29:16.892 INFO:tasks.workunit.client.0.trial031.stdout:Min latency(s): 0.00063351 2026-02-17T21:29:16.894 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:57: TEST_bluestore: echo 'after bench' 2026-02-17T21:29:16.894 INFO:tasks.workunit.client.0.trial031.stdout:after bench 2026-02-17T21:29:16.895 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: kill 87534 2026-02-17T21:29:16.895 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: sleep 1 2026-02-17T21:29:17.896 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: kill 87534 2026-02-17T21:29:17.896 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: sleep 1 2026-02-17T21:29:18.898 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: kill 87534 2026-02-17T21:29:18.899 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 60: kill: (87534) - No such process 2026-02-17T21:29:18.899 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:61: TEST_bluestore: ceph osd down 0 2026-02-17T21:29:19.094 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already down. 2026-02-17T21:29:19.103 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: kill 89808 2026-02-17T21:29:19.103 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: sleep 1 2026-02-17T21:29:20.104 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: kill 89808 2026-02-17T21:29:20.104 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 62: kill: (89808) - No such process 2026-02-17T21:29:20.104 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:63: TEST_bluestore: ceph osd down 1 2026-02-17T21:29:20.299 INFO:tasks.workunit.client.0.trial031.stderr:osd.1 is already down. 2026-02-17T21:29:20.308 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: kill 92661 2026-02-17T21:29:20.308 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: sleep 1 2026-02-17T21:29:21.309 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: kill 92661 2026-02-17T21:29:21.310 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: sleep 1 2026-02-17T21:29:22.312 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: kill 92661 2026-02-17T21:29:22.312 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 64: kill: (92661) - No such process 2026-02-17T21:29:22.312 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:65: TEST_bluestore: ceph osd down 2 2026-02-17T21:29:22.505 INFO:tasks.workunit.client.0.trial031.stderr:osd.2 is already down. 2026-02-17T21:29:22.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:66: TEST_bluestore: kill 95037 2026-02-17T21:29:22.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:66: TEST_bluestore: sleep 1 2026-02-17T21:29:23.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:66: TEST_bluestore: kill 95037 2026-02-17T21:29:23.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 66: kill: (95037) - No such process 2026-02-17T21:29:23.516 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:67: TEST_bluestore: ceph osd down 3 2026-02-17T21:29:23.726 INFO:tasks.workunit.client.0.trial031.stderr:osd.3 is already down. 2026-02-17T21:29:23.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:70: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-17T21:29:27.353 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:29:27.379 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:71: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-17T21:29:30.993 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:29:31.028 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:72: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-17T21:29:34.653 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:29:34.687 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:73: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-17T21:29:38.293 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:29:38.328 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:75: TEST_bluestore: truncate td/osd-bluefs-volume-ops/0/block -s 4294967296 2026-02-17T21:29:38.329 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:76: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 bluefs-bdev-expand 2026-02-17T21:29:38.348 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:29:40.550 INFO:tasks.workunit.client.0.trial031.stdout:0 : device size 0x20000000(512 MiB) : using 0xae00000(174 MiB) 2026-02-17T21:29:40.551 INFO:tasks.workunit.client.0.trial031.stdout:1 : device size 0x40000000(1 GiB) : using 0xc00000(12 MiB) 2026-02-17T21:29:40.551 INFO:tasks.workunit.client.0.trial031.stdout:2 : device size 0x100000000(4 GiB) : using 0xa8947000(2.6 GiB) 2026-02-17T21:29:40.551 INFO:tasks.workunit.client.0.trial031.stdout:Expanding DB/WAL... 2026-02-17T21:29:40.551 INFO:tasks.workunit.client.0.trial031.stdout:0 : nothing to do, skipped 2026-02-17T21:29:40.551 INFO:tasks.workunit.client.0.trial031.stdout:1 : nothing to do, skipped 2026-02-17T21:29:40.551 INFO:tasks.workunit.client.0.trial031.stdout:2 : Expanding to 0x100000000(4 GiB) 2026-02-17T21:29:40.552 INFO:tasks.workunit.client.0.trial031.stdout:2 : size updated to 0x100000000(4 GiB) 2026-02-17T21:29:44.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:77: TEST_bluestore: truncate td/osd-bluefs-volume-ops/1/block -s 11811160064 2026-02-17T21:29:44.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:78: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 bluefs-bdev-expand 2026-02-17T21:29:44.267 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:29:46.436 INFO:tasks.workunit.client.0.trial031.stdout:0 : device size 0x20000000(512 MiB) : using 0xaf00000(175 MiB) 2026-02-17T21:29:46.437 INFO:tasks.workunit.client.0.trial031.stdout:1 : device size 0x40000000(1 GiB) : using 0xc00000(12 MiB) 2026-02-17T21:29:46.437 INFO:tasks.workunit.client.0.trial031.stdout:2 : device size 0x2c0000000(11 GiB) : using 0x268a5a000(9.6 GiB) 2026-02-17T21:29:46.437 INFO:tasks.workunit.client.0.trial031.stdout:Expanding DB/WAL... 2026-02-17T21:29:46.437 INFO:tasks.workunit.client.0.trial031.stdout:0 : nothing to do, skipped 2026-02-17T21:29:46.437 INFO:tasks.workunit.client.0.trial031.stdout:1 : nothing to do, skipped 2026-02-17T21:29:46.437 INFO:tasks.workunit.client.0.trial031.stdout:2 : Expanding to 0x2c0000000(11 GiB) 2026-02-17T21:29:46.438 INFO:tasks.workunit.client.0.trial031.stdout:2 : size updated to 0x2c0000000(11 GiB) 2026-02-17T21:29:50.170 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:79: TEST_bluestore: truncate td/osd-bluefs-volume-ops/2/block -s 4295099392 2026-02-17T21:29:50.171 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:80: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 bluefs-bdev-expand 2026-02-17T21:29:50.184 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:29:52.406 INFO:tasks.workunit.client.0.trial031.stdout:0 : device size 0x20000000(512 MiB) : using 0xaf00000(175 MiB) 2026-02-17T21:29:52.407 INFO:tasks.workunit.client.0.trial031.stdout:1 : device size 0x40000000(1 GiB) : using 0xc00000(12 MiB) 2026-02-17T21:29:52.407 INFO:tasks.workunit.client.0.trial031.stdout:2 : device size 0x100020000(4.0 GiB) : using 0xa8872000(2.6 GiB) 2026-02-17T21:29:52.407 INFO:tasks.workunit.client.0.trial031.stdout:Expanding DB/WAL... 2026-02-17T21:29:52.407 INFO:tasks.workunit.client.0.trial031.stdout:0 : nothing to do, skipped 2026-02-17T21:29:52.407 INFO:tasks.workunit.client.0.trial031.stdout:1 : nothing to do, skipped 2026-02-17T21:29:52.407 INFO:tasks.workunit.client.0.trial031.stdout:2 : Expanding to 0x100020000(4.0 GiB) 2026-02-17T21:29:52.408 INFO:tasks.workunit.client.0.trial031.stdout:2 : size updated to 0x100020000(4.0 GiB) 2026-02-17T21:29:56.111 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:81: TEST_bluestore: truncate td/osd-bluefs-volume-ops/3/block -s 4293918720 2026-02-17T21:29:56.112 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:82: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 bluefs-bdev-expand 2026-02-17T21:29:56.130 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:29:58.296 INFO:tasks.workunit.client.0.trial031.stdout:0 : device size 0x20000000(512 MiB) : using 0xae00000(174 MiB) 2026-02-17T21:29:58.297 INFO:tasks.workunit.client.0.trial031.stdout:1 : device size 0x40000000(1 GiB) : using 0xc00000(12 MiB) 2026-02-17T21:29:58.297 INFO:tasks.workunit.client.0.trial031.stdout:2 : device size 0xfff00000(4.0 GiB) : using 0xa8853000(2.6 GiB) 2026-02-17T21:29:58.297 INFO:tasks.workunit.client.0.trial031.stdout:Expanding DB/WAL... 2026-02-17T21:29:58.297 INFO:tasks.workunit.client.0.trial031.stdout:0 : nothing to do, skipped 2026-02-17T21:29:58.297 INFO:tasks.workunit.client.0.trial031.stdout:1 : nothing to do, skipped 2026-02-17T21:29:58.297 INFO:tasks.workunit.client.0.trial031.stdout:2 : Expanding to 0xfff00000(4.0 GiB) 2026-02-17T21:29:58.298 INFO:tasks.workunit.client.0.trial031.stdout:2 : size updated to 0xfff00000(4.0 GiB) 2026-02-17T21:30:02.055 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:85: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-17T21:30:04.673 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:30:04.709 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:86: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-17T21:30:07.321 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:30:07.357 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:87: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-17T21:30:09.985 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:30:10.021 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:88: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-17T21:30:12.645 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:30:12.681 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:90: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 bluefs-bdev-sizes 2026-02-17T21:30:12.699 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:30:13.770 INFO:tasks.workunit.client.0.trial031.stdout:0 : device size 0x20000000(512 MiB) : using 0x800000(8 MiB) 2026-02-17T21:30:13.771 INFO:tasks.workunit.client.0.trial031.stdout:1 : device size 0x40000000(1 GiB) : using 0x3500000(53 MiB) 2026-02-17T21:30:13.771 INFO:tasks.workunit.client.0.trial031.stdout:2 : device size 0x100000000(4 GiB) : using 0x28947000(649 MiB) 2026-02-17T21:30:14.765 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:92: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.wal --dev-target td/osd-bluefs-volume-ops/0/block.db --command bluefs-bdev-migrate 2026-02-17T21:30:14.779 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:30:17.681 INFO:tasks.workunit.client.0.trial031.stdout: device removed:0 td/osd-bluefs-volume-ops/0/block.wal 2026-02-17T21:30:17.686 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:97: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-17T21:30:19.793 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:30:19.830 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:100: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block.db --dev-target td/osd-bluefs-volume-ops/1/block --command bluefs-bdev-migrate 2026-02-17T21:30:19.847 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:30:22.749 INFO:tasks.workunit.client.0.trial031.stdout: device removed:1 td/osd-bluefs-volume-ops/1/block.db 2026-02-17T21:30:22.755 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:105: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-17T21:30:24.865 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:30:24.902 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:108: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block.wal --devs-source td/osd-bluefs-volume-ops/2/block.db --dev-target td/osd-bluefs-volume-ops/2/block --command bluefs-bdev-migrate 2026-02-17T21:30:24.919 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:30:27.593 INFO:tasks.workunit.client.0.trial031.stdout: device removed:1 td/osd-bluefs-volume-ops/2/block.db 2026-02-17T21:30:27.593 INFO:tasks.workunit.client.0.trial031.stdout: device removed:0 td/osd-bluefs-volume-ops/2/block.wal 2026-02-17T21:30:27.600 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:114: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-17T21:30:29.209 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:30:29.245 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:117: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block.db --dev-target td/osd-bluefs-volume-ops/3/block.wal --command bluefs-bdev-migrate 2026-02-17T21:30:29.264 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:30:30.121 INFO:tasks.workunit.client.0.trial031.stderr:Migrate to WAL device isn't supported. 2026-02-17T21:30:30.123 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:123: TEST_bluestore: '[' 1 -eq 0 ']' 2026-02-17T21:30:30.123 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:126: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-17T21:30:32.737 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:30:32.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:129: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block.wal --dev-target td/osd-bluefs-volume-ops/3/block --command bluefs-bdev-migrate 2026-02-17T21:30:32.791 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:30:35.713 INFO:tasks.workunit.client.0.trial031.stdout: device removed:0 td/osd-bluefs-volume-ops/3/block.wal 2026-02-17T21:30:35.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:134: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-17T21:30:37.837 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:30:37.874 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:136: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block --dev-target td/osd-bluefs-volume-ops/3/block.db --command bluefs-bdev-migrate 2026-02-17T21:30:37.891 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:30:40.032 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:141: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-17T21:30:42.145 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:30:42.184 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:143: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 0 2026-02-17T21:30:42.184 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:30:42.184 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:30:42.184 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T21:30:42.184 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:30:42.184 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-17T21:30:42.184 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:30:42.184 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:30:42.184 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:30:42.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:30:42.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-17T21:30:42.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-17T21:30:42.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:30:42.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:30:42.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:30:42.185 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:30:42.185 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:30:42.185 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:30:42.185 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:30:42.185 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:30:42.185 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:30:42.186 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:30:42.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:30:42.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:30:42.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:30:42.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:30:42.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:30:42.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:30:42.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:30:42.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:30:42.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:30:42.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:30:42.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-17T21:30:42.188 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:30:42.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T21:30:42.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:30:42.189 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-17T21:30:42.190 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T21:30:42.190 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:30:42.190 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:30:42.190 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:30:42.204 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:30:42.201+0000 7f72165158c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:30:42.205 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:30:42.205+0000 7f72165158c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:30:42.206 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:30:42.205+0000 7f72165158c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:30:42.392 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T21:30:42.392 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:30:42.392 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:30:42.392 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:30:42.392 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:30:42.392 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:42.393 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:30:42.393 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:30:42.393 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:42.394 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:30:42.587 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:30:43.589 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:30:43.589 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:43.589 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:30:43.590 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:30:43.590 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:43.590 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:30:43.698 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:30:43.697+0000 7f72165158c0 -1 Falling back to public interface 2026-02-17T21:30:43.784 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:30:44.786 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:30:44.786 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:44.786 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:30:44.787 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:30:44.787 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:44.787 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:30:44.983 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:30:45.984 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:30:45.985 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:45.985 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:30:45.986 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:30:45.986 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:45.986 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:30:46.181 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:30:47.183 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:30:47.183 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:47.183 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:30:47.184 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:30:47.184 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:47.184 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:30:47.377 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:30:48.379 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:30:48.379 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:48.379 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:30:48.380 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:30:48.380 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:48.380 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:30:48.581 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:30:48.943 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:30:48.941+0000 7f72165158c0 -1 osd.0 23 log_to_monitors true 2026-02-17T21:30:49.583 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:30:49.583 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:49.584 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:30:49.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:30:49.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:49.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:30:49.786 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 32 up_thru 32 down_at 24 last_clean_interval [5,23) [v2:127.0.0.1:6802/315316553,v1:127.0.0.1:6803/315316553] [v2:127.0.0.1:6804/315316553,v1:127.0.0.1:6805/315316553] exists,up df635ca7-b5c9-471f-8ba7-205a3bf37cdb 2026-02-17T21:30:49.787 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:30:49.787 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:30:49.787 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:30:49.788 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:144: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-17T21:30:49.791 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:144: TEST_bluestore: osd_pid0=107507 2026-02-17T21:30:49.791 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:145: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 1 2026-02-17T21:30:49.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:30:49.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:30:49.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T21:30:49.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:30:49.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-17T21:30:49.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:30:49.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:30:49.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:30:49.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:30:49.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-17T21:30:49.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-17T21:30:49.793 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:30:49.793 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:30:49.793 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:30:49.793 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:30:49.793 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:30:49.793 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:30:49.793 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:30:49.793 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:30:49.793 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:30:49.793 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:30:49.793 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:30:49.793 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:30:49.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:30:49.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:30:49.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:30:49.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:30:49.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:30:49.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:30:49.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:30:49.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:30:49.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-17T21:30:49.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T21:30:49.795 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:30:49.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:30:49.796 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/1/whoami 2026-02-17T21:30:49.796 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T21:30:49.796 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:30:49.796 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:30:49.796 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:30:49.808 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:30:49.805+0000 7f0b0cdda8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:30:49.810 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:30:49.809+0000 7f0b0cdda8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:30:49.811 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:30:49.809+0000 7f0b0cdda8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:30:49.990 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T21:30:49.990 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:30:49.990 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:30:49.990 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:30:49.990 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:30:49.990 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:49.991 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:30:49.991 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:30:49.991 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:49.991 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:30:50.194 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:30:51.195 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:30:51.196 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:51.196 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:30:51.196 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:30:51.197 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:51.197 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:30:51.390 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:30:51.570 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:30:51.569+0000 7f0b0cdda8c0 -1 Falling back to public interface 2026-02-17T21:30:52.391 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:30:52.391 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:52.391 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:30:52.392 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:30:52.392 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:52.392 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:30:52.589 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:30:53.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:30:53.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:53.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:30:53.591 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:30:53.592 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:53.592 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:30:53.793 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:30:54.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:30:54.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:54.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:30:54.795 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:30:54.796 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:54.796 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:30:54.989 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:30:55.991 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:30:55.991 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:55.991 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:30:55.992 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:30:55.992 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:55.992 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:30:56.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:30:56.836 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:30:56.832+0000 7f0b0cdda8c0 -1 osd.1 25 log_to_monitors true 2026-02-17T21:30:57.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:30:57.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:57.188 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:30:57.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:30:57.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:57.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:30:57.394 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 35 up_thru 35 down_at 26 last_clean_interval [10,25) [v2:127.0.0.1:6810/3504147206,v1:127.0.0.1:6811/3504147206] [v2:127.0.0.1:6812/3504147206,v1:127.0.0.1:6813/3504147206] exists,up 3060462c-e701-42c9-891d-76dd092c903e 2026-02-17T21:30:57.395 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:30:57.395 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:30:57.395 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:30:57.396 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:146: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-17T21:30:57.399 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:146: TEST_bluestore: osd_pid1=108547 2026-02-17T21:30:57.399 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:147: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 2 2026-02-17T21:30:57.399 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:30:57.399 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:30:57.399 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T21:30:57.399 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:30:57.399 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-17T21:30:57.399 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:30:57.400 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:30:57.400 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:30:57.400 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:30:57.400 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-17T21:30:57.400 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-17T21:30:57.400 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:30:57.400 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:30:57.400 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:30:57.400 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:30:57.400 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:30:57.400 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:30:57.400 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:30:57.401 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:30:57.401 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:30:57.401 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:30:57.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:30:57.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:30:57.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:30:57.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:30:57.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:30:57.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:30:57.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:30:57.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:30:57.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:30:57.402 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:30:57.402 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-17T21:30:57.402 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:30:57.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T21:30:57.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:30:57.403 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/2/whoami 2026-02-17T21:30:57.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T21:30:57.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:30:57.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:30:57.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:30:57.423 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:30:57.420+0000 7f69f413e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:30:57.424 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:30:57.424+0000 7f69f413e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:30:57.426 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:30:57.424+0000 7f69f413e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:30:57.597 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T21:30:57.598 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:30:57.598 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:30:57.598 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:30:57.598 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:30:57.598 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:57.598 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:30:57.598 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:30:57.599 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:57.599 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:30:57.801 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:30:58.803 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:30:58.803 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:30:58.803 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:30:58.804 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:30:58.804 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:30:58.804 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:30:59.008 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:30:59.154 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:30:59.152+0000 7f69f413e8c0 -1 Falling back to public interface 2026-02-17T21:31:00.010 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:31:00.010 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:31:00.010 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:31:00.011 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:31:00.011 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:31:00.011 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:31:00.214 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:31:01.216 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:31:01.216 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:31:01.216 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:31:01.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:31:01.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:31:01.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:31:01.421 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:31:02.422 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:31:02.423 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:31:02.423 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:31:02.424 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:31:02.424 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:31:02.424 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:31:02.627 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:31:03.621 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:31:03.620+0000 7f69f413e8c0 -1 osd.2 27 log_to_monitors true 2026-02-17T21:31:03.628 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:31:03.628 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:31:03.628 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:31:03.629 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:31:03.629 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:31:03.629 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:31:03.833 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 39 up_thru 26 down_at 28 last_clean_interval [14,27) [v2:127.0.0.1:6818/510209066,v1:127.0.0.1:6819/510209066] [v2:127.0.0.1:6820/510209066,v1:127.0.0.1:6821/510209066] exists,up 3d0ce96d-e4ec-4623-a2b6-711f320be599 2026-02-17T21:31:03.834 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:31:03.834 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:31:03.834 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:31:03.835 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:148: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-17T21:31:03.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:148: TEST_bluestore: osd_pid2=109587 2026-02-17T21:31:03.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:149: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 3 2026-02-17T21:31:03.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:31:03.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:31:03.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T21:31:03.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:31:03.838 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-17T21:31:03.839 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:31:03.839 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:31:03.839 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:31:03.839 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:31:03.839 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-17T21:31:03.839 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-17T21:31:03.839 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:31:03.839 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:31:03.839 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:31:03.839 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:31:03.839 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:31:03.840 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:31:03.840 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:31:03.840 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:31:03.840 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:31:03.840 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:31:03.840 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:31:03.840 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:31:03.840 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:31:03.840 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:31:03.840 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:31:03.840 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:31:03.840 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:31:03.841 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:31:03.841 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:31:03.841 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:31:03.841 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-17T21:31:03.841 INFO:tasks.workunit.client.0.trial031.stdout:start osd.3 2026-02-17T21:31:03.842 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T21:31:03.842 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:31:03.842 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/3/whoami 2026-02-17T21:31:03.842 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T21:31:03.843 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:31:03.843 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:31:03.843 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:31:03.860 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:31:03.860+0000 7fc715e368c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:31:03.862 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:31:03.860+0000 7fc715e368c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:31:03.863 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:31:03.860+0000 7fc715e368c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:31:04.042 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T21:31:04.042 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:31:04.042 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T21:31:04.042 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:31:04.043 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:31:04.043 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:31:04.043 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:31:04.044 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:31:04.044 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:31:04.044 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:31:04.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:31:05.240 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:31:05.240 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:31:05.241 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:31:05.241 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:31:05.241 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:31:05.241 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:31:05.436 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:31:05.618 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:31:05.616+0000 7fc715e368c0 -1 Falling back to public interface 2026-02-17T21:31:06.437 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:31:06.437 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:31:06.438 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:31:06.438 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:31:06.439 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:31:06.439 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:31:06.634 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:31:07.635 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:31:07.635 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:31:07.636 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:31:07.636 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:31:07.637 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:31:07.637 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:31:07.834 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:31:08.836 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:31:08.836 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:31:08.836 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:31:08.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:31:08.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:31:08.837 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:31:09.039 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:31:10.040 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:31:10.040 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:31:10.041 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:31:10.041 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:31:10.041 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:31:10.042 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:31:10.236 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:31:10.896 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:31:10.892+0000 7fc715e368c0 -1 osd.3 29 log_to_monitors true 2026-02-17T21:31:11.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:31:11.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:31:11.238 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:31:11.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:31:11.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:31:11.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:31:11.435 INFO:tasks.workunit.client.0.trial031.stdout:osd.3 up in weight 1 up_from 42 up_thru 42 down_at 30 last_clean_interval [19,29) [v2:127.0.0.1:6826/349500975,v1:127.0.0.1:6827/349500975] [v2:127.0.0.1:6828/349500975,v1:127.0.0.1:6829/349500975] exists,up 3c7e5245-677b-4faf-ac5c-593720b47b50 2026-02-17T21:31:11.436 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:31:11.436 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:31:11.436 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:31:11.437 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:150: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-17T21:31:11.437 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:150: TEST_bluestore: osd_pid3=110597 2026-02-17T21:31:11.437 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:152: TEST_bluestore: wait_for_clean 2026-02-17T21:31:11.438 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:31:11.438 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:31:11.438 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:31:11.439 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T21:31:11.439 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:31:11.439 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:31:11.439 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:31:11.439 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:31:11.439 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:31:11.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T21:31:11.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:31:11.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:31:11.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:31:11.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:31:11.532 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:31:11.741 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:31:11.741 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:31:11.741 INFO:tasks.workunit.client.0.trial031.stderr:2 2026-02-17T21:31:11.742 INFO:tasks.workunit.client.0.trial031.stderr:3' 2026-02-17T21:31:11.742 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:31:11.742 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:31:11.742 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:31:11.816 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953479 2026-02-17T21:31:11.816 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953479 2026-02-17T21:31:11.816 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-137438953479' 2026-02-17T21:31:11.816 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:31:11.816 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:31:11.892 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855365 2026-02-17T21:31:11.892 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855365 2026-02-17T21:31:11.892 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-137438953479 1-150323855365' 2026-02-17T21:31:11.892 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:31:11.893 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:31:11.969 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724548 2026-02-17T21:31:11.969 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724548 2026-02-17T21:31:11.969 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-137438953479 1-150323855365 2-167503724548' 2026-02-17T21:31:11.969 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:31:11.970 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T21:31:12.044 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626435 2026-02-17T21:31:12.044 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626435 2026-02-17T21:31:12.044 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-137438953479 1-150323855365 2-167503724548 3-180388626435' 2026-02-17T21:31:12.044 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:31:12.045 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-137438953479 2026-02-17T21:31:12.045 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:31:12.046 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:31:12.047 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-137438953479 2026-02-17T21:31:12.047 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:31:12.049 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953479 2026-02-17T21:31:12.049 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 137438953479' 2026-02-17T21:31:12.049 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 137438953479 2026-02-17T21:31:12.049 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:31:12.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953477 -lt 137438953479 2026-02-17T21:31:12.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:31:13.250 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:31:13.251 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:31:13.447 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953479 -lt 137438953479 2026-02-17T21:31:13.447 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:31:13.448 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-150323855365 2026-02-17T21:31:13.448 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:31:13.449 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:31:13.450 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-150323855365 2026-02-17T21:31:13.450 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:31:13.452 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855365 2026-02-17T21:31:13.452 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 150323855365' 2026-02-17T21:31:13.452 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 150323855365 2026-02-17T21:31:13.452 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:31:13.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855366 -lt 150323855365 2026-02-17T21:31:13.668 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:31:13.669 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-167503724548 2026-02-17T21:31:13.669 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:31:13.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:31:13.671 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-167503724548 2026-02-17T21:31:13.671 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:31:13.672 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724548 2026-02-17T21:31:13.673 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 167503724548' 2026-02-17T21:31:13.673 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 167503724548 2026-02-17T21:31:13.673 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:31:13.876 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724548 -lt 167503724548 2026-02-17T21:31:13.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:31:13.877 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-180388626435 2026-02-17T21:31:13.877 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:31:13.878 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T21:31:13.879 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-180388626435 2026-02-17T21:31:13.879 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:31:13.881 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626435 2026-02-17T21:31:13.881 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 180388626435' 2026-02-17T21:31:13.881 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.3 seq 180388626435 2026-02-17T21:31:13.881 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T21:31:14.077 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626435 -lt 180388626435 2026-02-17T21:31:14.078 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:31:14.078 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:31:14.078 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:31:14.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 == 0 2026-02-17T21:31:14.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:31:14.335 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:31:14.335 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:31:14.335 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:31:14.335 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:31:14.335 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:31:14.335 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:31:14.530 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=16 2026-02-17T21:31:14.530 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:31:14.531 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:31:14.531 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:31:14.787 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 16 = 16 2026-02-17T21:31:14.788 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T21:31:14.788 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T21:31:14.788 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:155: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-17T21:31:14.805 INFO:tasks.workunit.client.0.trial031.stdout:hints = 1 2026-02-17T21:31:14.805 INFO:tasks.workunit.client.0.trial031.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-17T21:31:14.805 INFO:tasks.workunit.client.0.trial031.stdout:Object prefix: benchmark_data_trial031_112005 2026-02-17T21:31:14.805 INFO:tasks.workunit.client.0.trial031.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-17T21:31:14.805 INFO:tasks.workunit.client.0.trial031.stdout: 0 0 0 0 0 0 - 0 2026-02-17T21:31:15.805 INFO:tasks.workunit.client.0.trial031.stdout: 1 16 8542 8526 33.302 33.3047 0.00214423 0.00187372 2026-02-17T21:31:16.805 INFO:tasks.workunit.client.0.trial031.stdout: 2 16 16929 16913 33.0306 32.7617 0.00195267 0.00188986 2026-02-17T21:31:17.805 INFO:tasks.workunit.client.0.trial031.stdout: 3 15 23635 23620 30.7528 26.1992 0.00239856 0.00203015 2026-02-17T21:31:18.805 INFO:tasks.workunit.client.0.trial031.stdout: 4 15 31821 31806 31.0582 31.9766 0.00221798 0.00201053 2026-02-17T21:31:19.805 INFO:tasks.workunit.client.0.trial031.stdout: 5 16 40068 40052 31.2883 32.2109 0.00167905 0.00199588 2026-02-17T21:31:20.805 INFO:tasks.workunit.client.0.trial031.stdout: 6 16 48270 48254 31.413 32.0391 0.00187776 0.00198809 2026-02-17T21:31:21.805 INFO:tasks.workunit.client.0.trial031.stdout: 7 16 56470 56454 31.5009 32.0312 0.00187643 0.00198242 2026-02-17T21:31:22.805 INFO:tasks.workunit.client.0.trial031.stdout: 8 16 64810 64794 31.6352 32.5781 0.00173832 0.00197403 2026-02-17T21:31:23.805 INFO:tasks.workunit.client.0.trial031.stdout: 9 16 66226 66210 28.7347 5.53125 0.148582 0.00213332 2026-02-17T21:31:24.805 INFO:tasks.workunit.client.0.trial031.stdout: 10 16 74432 74416 29.0665 32.0547 0.00168307 0.00214868 2026-02-17T21:31:25.806 INFO:tasks.workunit.client.0.trial031.stdout: 11 16 82936 82920 29.4438 33.2188 0.00201841 0.00212112 2026-02-17T21:31:26.806 INFO:tasks.workunit.client.0.trial031.stdout: 12 15 91454 91439 29.763 33.2773 0.00515627 0.00209828 2026-02-17T21:31:27.806 INFO:tasks.workunit.client.0.trial031.stdout: 13 16 99919 99903 30.0167 33.0625 0.00173601 0.00208069 2026-02-17T21:31:28.806 INFO:tasks.workunit.client.0.trial031.stdout: 14 16 108426 108410 30.246 33.2305 0.00164191 0.00206487 2026-02-17T21:31:29.806 INFO:tasks.workunit.client.0.trial031.stdout: 15 16 110814 110798 28.8515 9.32812 0.00163107 0.00216478 2026-02-17T21:31:30.806 INFO:tasks.workunit.client.0.trial031.stdout: 16 16 119251 119235 29.1079 32.957 0.00159108 0.00214571 2026-02-17T21:31:31.806 INFO:tasks.workunit.client.0.trial031.stdout: 17 16 127601 127585 29.3142 32.6172 0.00138024 0.00213061 2026-02-17T21:31:32.806 INFO:tasks.workunit.client.0.trial031.stdout: 18 16 135997 135981 29.5076 32.7969 0.00143913 0.00211664 2026-02-17T21:31:33.806 INFO:tasks.workunit.client.0.trial031.stdout: 19 16 144241 144225 29.6493 32.2031 0.00185632 0.00210648 2026-02-17T21:31:34.806 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:31:34.810014+0000 min lat: 0.000742422 max lat: 0.239886 avg lat: 0.00210053 2026-02-17T21:31:34.806 INFO:tasks.workunit.client.0.trial031.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-17T21:31:34.806 INFO:tasks.workunit.client.0.trial031.stdout: 20 16 152098 152082 29.7013 30.6914 0.0423211 0.00210053 2026-02-17T21:31:35.806 INFO:tasks.workunit.client.0.trial031.stdout: 21 16 152228 152212 28.3111 0.507812 0.135062 0.00219816 2026-02-17T21:31:36.806 INFO:tasks.workunit.client.0.trial031.stdout: 22 16 157229 157213 27.9121 19.5352 0.00147865 0.00223772 2026-02-17T21:31:37.806 INFO:tasks.workunit.client.0.trial031.stdout: 23 16 165512 165496 28.1052 32.3555 0.00219593 0.00222234 2026-02-17T21:31:38.806 INFO:tasks.workunit.client.0.trial031.stdout: 24 16 173884 173868 28.2967 32.7031 0.00165486 0.00220732 2026-02-17T21:31:39.806 INFO:tasks.workunit.client.0.trial031.stdout: 25 16 182272 182256 28.4754 32.7656 0.00148639 0.00219345 2026-02-17T21:31:40.807 INFO:tasks.workunit.client.0.trial031.stdout: 26 16 190686 190670 28.6442 32.8672 0.00212299 0.00218049 2026-02-17T21:31:41.807 INFO:tasks.workunit.client.0.trial031.stdout: 27 15 198947 198932 28.7785 32.2734 0.00216515 0.00217034 2026-02-17T21:31:42.807 INFO:tasks.workunit.client.0.trial031.stdout: 28 16 207307 207291 28.9168 32.6523 0.0018011 0.00215995 2026-02-17T21:31:43.807 INFO:tasks.workunit.client.0.trial031.stdout: 29 16 215557 215541 29.0308 32.2266 0.00239032 0.00215145 2026-02-17T21:31:44.807 INFO:tasks.workunit.client.0.trial031.stdout:Total time run: 30.0012 2026-02-17T21:31:44.807 INFO:tasks.workunit.client.0.trial031.stdout:Total writes made: 223891 2026-02-17T21:31:44.807 INFO:tasks.workunit.client.0.trial031.stdout:Write size: 4096 2026-02-17T21:31:44.807 INFO:tasks.workunit.client.0.trial031.stdout:Object size: 4096 2026-02-17T21:31:44.807 INFO:tasks.workunit.client.0.trial031.stdout:Bandwidth (MB/sec): 29.1513 2026-02-17T21:31:44.808 INFO:tasks.workunit.client.0.trial031.stdout:Stddev Bandwidth: 8.77236 2026-02-17T21:31:44.808 INFO:tasks.workunit.client.0.trial031.stdout:Max bandwidth (MB/sec): 33.3047 2026-02-17T21:31:44.808 INFO:tasks.workunit.client.0.trial031.stdout:Min bandwidth (MB/sec): 0.507812 2026-02-17T21:31:44.808 INFO:tasks.workunit.client.0.trial031.stdout:Average IOPS: 7462 2026-02-17T21:31:44.808 INFO:tasks.workunit.client.0.trial031.stdout:Stddev IOPS: 2245.72 2026-02-17T21:31:44.808 INFO:tasks.workunit.client.0.trial031.stdout:Max IOPS: 8526 2026-02-17T21:31:44.808 INFO:tasks.workunit.client.0.trial031.stdout:Min IOPS: 130 2026-02-17T21:31:44.808 INFO:tasks.workunit.client.0.trial031.stdout:Average Latency(s): 0.00214258 2026-02-17T21:31:44.808 INFO:tasks.workunit.client.0.trial031.stdout:Stddev Latency(s): 0.00528489 2026-02-17T21:31:44.808 INFO:tasks.workunit.client.0.trial031.stdout:Max latency(s): 0.32518 2026-02-17T21:31:44.808 INFO:tasks.workunit.client.0.trial031.stdout:Min latency(s): 0.000742422 2026-02-17T21:31:44.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:158: TEST_bluestore: kill 107507 2026-02-17T21:31:44.810 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:158: TEST_bluestore: sleep 1 2026-02-17T21:31:45.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:158: TEST_bluestore: kill 107507 2026-02-17T21:31:45.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 158: kill: (107507) - No such process 2026-02-17T21:31:45.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:159: TEST_bluestore: ceph osd down 0 2026-02-17T21:31:46.017 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already down. 2026-02-17T21:31:46.027 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:160: TEST_bluestore: kill 108547 2026-02-17T21:31:46.027 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:160: TEST_bluestore: sleep 1 2026-02-17T21:31:47.028 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:160: TEST_bluestore: kill 108547 2026-02-17T21:31:47.028 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 160: kill: (108547) - No such process 2026-02-17T21:31:47.029 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:161: TEST_bluestore: ceph osd down 1 2026-02-17T21:31:47.228 INFO:tasks.workunit.client.0.trial031.stderr:osd.1 is already down. 2026-02-17T21:31:47.237 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:162: TEST_bluestore: kill 109587 2026-02-17T21:31:47.238 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:162: TEST_bluestore: sleep 1 2026-02-17T21:31:48.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:162: TEST_bluestore: kill 109587 2026-02-17T21:31:48.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 162: kill: (109587) - No such process 2026-02-17T21:31:48.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:163: TEST_bluestore: ceph osd down 2 2026-02-17T21:31:48.443 INFO:tasks.workunit.client.0.trial031.stderr:osd.2 is already down. 2026-02-17T21:31:48.452 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:164: TEST_bluestore: kill 110597 2026-02-17T21:31:48.452 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:164: TEST_bluestore: sleep 1 2026-02-17T21:31:49.453 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:164: TEST_bluestore: kill 110597 2026-02-17T21:31:49.454 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 164: kill: (110597) - No such process 2026-02-17T21:31:49.454 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:165: TEST_bluestore: ceph osd down 3 2026-02-17T21:31:49.741 INFO:tasks.workunit.client.0.trial031.stderr:osd.3 is already down. 2026-02-17T21:31:49.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:168: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-17T21:31:53.632 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:31:53.683 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:170: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/0/wal count=512 bs=1M 2026-02-17T21:31:54.006 INFO:tasks.workunit.client.0.trial031.stderr:512+0 records in 2026-02-17T21:31:54.007 INFO:tasks.workunit.client.0.trial031.stderr:512+0 records out 2026-02-17T21:31:54.007 INFO:tasks.workunit.client.0.trial031.stderr:536870912 bytes (537 MB, 512 MiB) copied, 0.321999 s, 1.7 GB/s 2026-02-17T21:31:54.007 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:171: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --dev-target td/osd-bluefs-volume-ops/0/wal --command bluefs-bdev-new-wal 2026-02-17T21:31:54.025 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:31:57.908 INFO:tasks.workunit.client.0.trial031.stdout:WAL device added /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/0/wal 2026-02-17T21:31:57.930 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:175: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-17T21:32:02.336 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:32:02.388 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:178: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-17T21:32:06.248 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:32:06.300 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:180: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/1/db count=1024 bs=1M 2026-02-17T21:32:06.931 INFO:tasks.workunit.client.0.trial031.stderr:1024+0 records in 2026-02-17T21:32:06.932 INFO:tasks.workunit.client.0.trial031.stderr:1024+0 records out 2026-02-17T21:32:06.932 INFO:tasks.workunit.client.0.trial031.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.629841 s, 1.7 GB/s 2026-02-17T21:32:06.932 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:181: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --dev-target td/osd-bluefs-volume-ops/1/db --command bluefs-bdev-new-db 2026-02-17T21:32:06.949 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:32:11.572 INFO:tasks.workunit.client.0.trial031.stdout:DB device added /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/1/db 2026-02-17T21:32:11.595 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:185: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block --dev-target td/osd-bluefs-volume-ops/1/block.db --command bluefs-bdev-migrate 2026-02-17T21:32:11.610 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:32:16.056 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:190: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-17T21:32:20.420 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:32:20.468 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:193: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-17T21:32:23.828 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:32:23.876 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:195: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --command bluefs-bdev-new-db 2026-02-17T21:32:23.894 INFO:tasks.workunit.client.0.trial031.stdout:NOTICE: --dev-target option omitted, will allocate as a file 2026-02-17T21:32:23.894 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:32:26.736 INFO:tasks.workunit.client.0.trial031.stdout:DB device added 2026-02-17T21:32:26.759 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:198: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --command bluefs-bdev-new-wal 2026-02-17T21:32:26.774 INFO:tasks.workunit.client.0.trial031.stdout:NOTICE: --dev-target option omitted, will allocate as a file 2026-02-17T21:32:26.774 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:32:30.412 INFO:tasks.workunit.client.0.trial031.stdout:WAL device added 2026-02-17T21:32:30.434 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:201: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block --dev-target td/osd-bluefs-volume-ops/2/block.db --command bluefs-bdev-migrate 2026-02-17T21:32:30.450 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:32:34.903 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:206: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-17T21:32:39.261 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:32:39.308 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:209: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-17T21:32:43.197 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:32:43.244 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:211: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --command bluefs-bdev-new-wal 2026-02-17T21:32:43.258 INFO:tasks.workunit.client.0.trial031.stdout:NOTICE: --dev-target option omitted, will allocate as a file 2026-02-17T21:32:43.258 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:32:46.897 INFO:tasks.workunit.client.0.trial031.stdout:WAL device added 2026-02-17T21:32:46.920 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:214: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block.db --dev-target td/osd-bluefs-volume-ops/3/block --command bluefs-bdev-migrate 2026-02-17T21:32:46.935 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:32:51.609 INFO:tasks.workunit.client.0.trial031.stdout: device removed:1 td/osd-bluefs-volume-ops/3/block.db 2026-02-17T21:32:51.633 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:219: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-17T21:32:55.493 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:32:55.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:221: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 0 2026-02-17T21:32:55.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:32:55.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:32:55.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T21:32:55.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:32:55.544 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-17T21:32:55.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:32:55.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:32:55.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:32:55.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:32:55.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-17T21:32:55.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-17T21:32:55.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:32:55.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:32:55.545 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:32:55.545 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:32:55.545 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:32:55.545 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:32:55.546 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:32:55.546 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:32:55.546 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:32:55.546 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:32:55.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:32:55.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:32:55.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:32:55.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:32:55.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:32:55.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:32:55.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:32:55.547 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:32:55.547 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:32:55.547 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:32:55.547 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-17T21:32:55.547 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:32:55.548 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T21:32:55.548 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:32:55.548 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-17T21:32:55.549 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T21:32:55.549 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:32:55.549 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:32:55.549 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:32:55.565 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:32:55.565+0000 7f3e631e28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:32:55.566 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:32:55.565+0000 7f3e631e28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:32:55.568 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:32:55.565+0000 7f3e631e28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:32:55.744 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T21:32:55.744 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:32:55.744 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:32:55.744 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:32:55.744 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:32:55.744 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:32:55.745 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:32:55.745 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:32:55.745 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:32:55.745 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:32:55.947 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:32:56.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:32:56.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:32:56.950 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:32:56.950 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:32:56.950 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:32:56.950 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:32:57.149 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:32:57.586 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:32:57.585+0000 7f3e631e28c0 -1 Falling back to public interface 2026-02-17T21:32:58.151 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:32:58.151 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:32:58.151 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:32:58.152 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:32:58.152 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:32:58.152 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:32:58.351 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:32:59.352 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:32:59.352 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:32:59.352 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:32:59.353 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:32:59.353 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:32:59.353 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:32:59.557 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:00.559 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:00.559 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:00.560 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:33:00.560 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:33:00.560 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:00.560 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:33:00.755 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:01.757 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:01.757 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:01.757 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:33:01.757 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:33:01.758 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:01.758 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:33:01.964 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:02.965 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:02.966 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:02.966 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:33:02.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:33:02.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:02.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:33:03.165 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:04.167 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:04.167 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:04.167 INFO:tasks.workunit.client.0.trial031.stdout:7 2026-02-17T21:33:04.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-17T21:33:04.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:04.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:33:04.366 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:05.368 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:05.368 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:05.368 INFO:tasks.workunit.client.0.trial031.stdout:8 2026-02-17T21:33:05.369 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-17T21:33:05.369 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:05.369 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:33:05.570 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:06.571 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:06.572 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:06.572 INFO:tasks.workunit.client.0.trial031.stdout:9 2026-02-17T21:33:06.573 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-17T21:33:06.573 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:06.573 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:33:06.770 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:06.871 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:06.868+0000 7f3e631e28c0 -1 osd.0 43 log_to_monitors true 2026-02-17T21:33:07.771 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:07.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:07.772 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-17T21:33:07.772 INFO:tasks.workunit.client.0.trial031.stdout:10 2026-02-17T21:33:07.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:07.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:33:07.974 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 52 up_thru 52 down_at 44 last_clean_interval [32,43) [v2:127.0.0.1:6802/3030749983,v1:127.0.0.1:6803/3030749983] [v2:127.0.0.1:6804/3030749983,v1:127.0.0.1:6805/3030749983] exists,up df635ca7-b5c9-471f-8ba7-205a3bf37cdb 2026-02-17T21:33:07.975 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:33:07.975 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:33:07.975 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:33:07.975 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:222: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-17T21:33:07.978 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:222: TEST_bluestore: osd_pid0=118156 2026-02-17T21:33:07.978 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:223: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 1 2026-02-17T21:33:07.979 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:33:07.979 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:33:07.979 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T21:33:07.979 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:33:07.979 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-17T21:33:07.979 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:33:07.979 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:33:07.979 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:33:07.979 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:33:07.979 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-17T21:33:07.980 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-17T21:33:07.980 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:33:07.980 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:33:07.980 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:33:07.980 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:33:07.980 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:33:07.980 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:33:07.980 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:33:07.980 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:33:07.980 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:33:07.980 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:33:07.980 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:33:07.981 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:33:07.981 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:33:07.981 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:33:07.981 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:33:07.981 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:33:07.981 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:33:07.981 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:33:07.981 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:33:07.981 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:33:07.981 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-17T21:33:07.982 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:33:07.983 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T21:33:07.983 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:33:07.983 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/1/whoami 2026-02-17T21:33:07.983 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T21:33:07.983 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:33:07.983 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:33:07.984 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:33:07.997 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:07.996+0000 7fbef7fc78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:33:07.998 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:07.996+0000 7fbef7fc78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:33:07.999 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:07.996+0000 7fbef7fc78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:33:08.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T21:33:08.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:33:08.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:33:08.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:33:08.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:33:08.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:08.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:33:08.187 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:33:08.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:08.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:33:08.383 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:09.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:09.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:09.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:33:09.386 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:33:09.386 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:09.386 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:33:09.582 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:09.706 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:09.704+0000 7fbef7fc78c0 -1 Falling back to public interface 2026-02-17T21:33:10.583 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:10.584 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:10.584 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:33:10.585 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:33:10.585 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:10.585 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:33:10.788 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:11.789 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:11.790 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:11.790 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:33:11.791 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:33:11.791 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:11.791 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:33:11.989 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:12.991 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:12.991 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:12.991 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:33:12.992 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:33:12.992 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:12.992 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:33:13.192 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:14.194 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:14.194 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:14.194 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:33:14.194 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:33:14.195 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:14.195 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:33:14.394 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:15.396 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:15.396 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:15.396 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:33:15.397 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:33:15.397 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:15.397 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:33:15.589 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:16.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:16.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:16.591 INFO:tasks.workunit.client.0.trial031.stdout:7 2026-02-17T21:33:16.592 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-17T21:33:16.592 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:16.592 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:33:16.796 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:17.798 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:17.798 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:17.798 INFO:tasks.workunit.client.0.trial031.stdout:8 2026-02-17T21:33:17.799 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-17T21:33:17.799 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:17.799 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:33:18.001 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:18.653 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:18.652+0000 7fbef7fc78c0 -1 osd.1 45 log_to_monitors true 2026-02-17T21:33:19.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:19.002 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:19.003 INFO:tasks.workunit.client.0.trial031.stdout:9 2026-02-17T21:33:19.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-17T21:33:19.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:19.003 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:33:19.202 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 55 up_thru 55 down_at 46 last_clean_interval [35,45) [v2:127.0.0.1:6810/2851901111,v1:127.0.0.1:6811/2851901111] [v2:127.0.0.1:6812/2851901111,v1:127.0.0.1:6813/2851901111] exists,up 3060462c-e701-42c9-891d-76dd092c903e 2026-02-17T21:33:19.203 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:33:19.203 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:33:19.203 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:33:19.203 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:224: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:224: TEST_bluestore: osd_pid1=119301 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:225: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 2 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:33:19.205 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:33:19.206 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:33:19.206 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:33:19.206 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:33:19.206 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:33:19.206 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:33:19.206 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:33:19.206 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:33:19.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:33:19.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:33:19.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:33:19.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:33:19.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:33:19.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:33:19.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:33:19.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:33:19.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:33:19.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:33:19.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-17T21:33:19.207 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:33:19.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T21:33:19.208 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:33:19.208 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/2/whoami 2026-02-17T21:33:19.209 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T21:33:19.209 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:33:19.209 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:33:19.209 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:33:19.225 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:19.224+0000 7f73c489d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:33:19.227 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:19.224+0000 7f73c489d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:33:19.228 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:19.224+0000 7f73c489d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:33:19.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T21:33:19.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:33:19.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:33:19.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:33:19.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:33:19.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:19.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:33:19.409 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:33:19.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:19.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:33:19.617 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:20.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:20.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:20.619 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:33:20.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:33:20.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:20.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:33:20.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:20.982 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:20.980+0000 7f73c489d8c0 -1 Falling back to public interface 2026-02-17T21:33:21.822 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:21.823 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:21.823 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:33:21.823 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:33:21.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:21.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:33:22.024 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:23.025 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:23.026 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:23.026 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:33:23.027 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:33:23.027 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:23.027 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:33:23.225 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:24.227 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:24.227 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:24.227 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:33:24.228 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:33:24.228 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:24.228 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:33:24.420 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:25.422 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:25.422 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:25.422 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:33:25.423 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:33:25.423 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:25.423 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:33:25.617 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:26.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:26.619 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:26.620 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:33:26.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:33:26.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:26.620 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:33:26.811 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:27.813 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:27.813 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:27.814 INFO:tasks.workunit.client.0.trial031.stdout:7 2026-02-17T21:33:27.814 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-17T21:33:27.814 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:27.815 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:33:28.010 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:29.011 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:29.011 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:29.012 INFO:tasks.workunit.client.0.trial031.stdout:8 2026-02-17T21:33:29.012 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-17T21:33:29.012 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:29.012 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:33:29.214 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:29.989 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:29.992+0000 7f73c489d8c0 -1 osd.2 47 log_to_monitors true 2026-02-17T21:33:30.215 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:30.216 INFO:tasks.workunit.client.0.trial031.stdout:9 2026-02-17T21:33:30.216 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:30.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-17T21:33:30.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:30.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:33:30.410 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 58 up_thru 58 down_at 48 last_clean_interval [39,47) [v2:127.0.0.1:6818/4037730732,v1:127.0.0.1:6819/4037730732] [v2:127.0.0.1:6820/4037730732,v1:127.0.0.1:6821/4037730732] exists,up 3d0ce96d-e4ec-4623-a2b6-711f320be599 2026-02-17T21:33:30.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:33:30.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:33:30.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:33:30.411 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:226: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-17T21:33:30.414 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:226: TEST_bluestore: osd_pid2=120421 2026-02-17T21:33:30.414 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:227: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 3 2026-02-17T21:33:30.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:33:30.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:33:30.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T21:33:30.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:33:30.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-17T21:33:30.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:33:30.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:33:30.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:33:30.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:33:30.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-17T21:33:30.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-17T21:33:30.416 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:33:30.416 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:33:30.416 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:33:30.416 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:33:30.416 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:33:30.416 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:33:30.416 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:33:30.416 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:33:30.416 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:33:30.416 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:33:30.416 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:33:30.416 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:33:30.417 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:33:30.417 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:33:30.417 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:33:30.417 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:33:30.417 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:33:30.417 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:33:30.417 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:33:30.417 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:33:30.417 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-17T21:33:30.418 INFO:tasks.workunit.client.0.trial031.stdout:start osd.3 2026-02-17T21:33:30.419 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T21:33:30.419 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:33:30.419 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/3/whoami 2026-02-17T21:33:30.419 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T21:33:30.419 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:33:30.419 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:33:30.419 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:33:30.433 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:30.436+0000 7fc9121c88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:33:30.434 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:30.436+0000 7fc9121c88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:33:30.435 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:30.436+0000 7fc9121c88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:33:30.612 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T21:33:30.612 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:33:30.612 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T21:33:30.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:33:30.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:33:30.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:30.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:33:30.613 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:33:30.614 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:30.614 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:33:30.812 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:31.813 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:31.813 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:31.814 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:33:31.814 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:33:31.815 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:31.815 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:33:31.950 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:31.951+0000 7fc9121c88c0 -1 Falling back to public interface 2026-02-17T21:33:32.016 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:33.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:33.018 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:33.019 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:33:33.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:33:33.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:33.020 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:33:33.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:34.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:34.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:34.220 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:33:34.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:33:34.221 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:34.221 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:33:34.426 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:35.428 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:35.428 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:35.428 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:33:35.429 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:33:35.429 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:35.429 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:33:35.625 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:36.626 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:36.627 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:36.627 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:33:36.627 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:33:36.628 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:36.628 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:33:36.821 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:37.822 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:37.823 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:37.823 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:33:37.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:33:37.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:37.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:33:38.020 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:39.021 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:39.021 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:39.021 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-17T21:33:39.021 INFO:tasks.workunit.client.0.trial031.stdout:7 2026-02-17T21:33:39.022 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:39.022 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:33:39.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:40.221 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:40.221 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:40.222 INFO:tasks.workunit.client.0.trial031.stdout:8 2026-02-17T21:33:40.222 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-17T21:33:40.222 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:40.222 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:33:40.437 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:33:40.693 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:33:40.695+0000 7fc9121c88c0 -1 osd.3 49 log_to_monitors true 2026-02-17T21:33:41.439 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:33:41.439 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:33:41.439 INFO:tasks.workunit.client.0.trial031.stdout:9 2026-02-17T21:33:41.440 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-17T21:33:41.440 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:33:41.440 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:33:41.643 INFO:tasks.workunit.client.0.trial031.stdout:osd.3 up in weight 1 up_from 61 up_thru 61 down_at 50 last_clean_interval [42,49) [v2:127.0.0.1:6826/3538790395,v1:127.0.0.1:6827/3538790395] [v2:127.0.0.1:6828/3538790395,v1:127.0.0.1:6829/3538790395] exists,up 3c7e5245-677b-4faf-ac5c-593720b47b50 2026-02-17T21:33:41.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:33:41.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:33:41.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:33:41.644 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:228: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-17T21:33:41.645 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:228: TEST_bluestore: osd_pid3=121541 2026-02-17T21:33:41.645 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:231: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-17T21:33:41.663 INFO:tasks.workunit.client.0.trial031.stdout:hints = 1 2026-02-17T21:33:41.663 INFO:tasks.workunit.client.0.trial031.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-17T21:33:41.664 INFO:tasks.workunit.client.0.trial031.stdout:Object prefix: benchmark_data_trial031_122641 2026-02-17T21:33:41.664 INFO:tasks.workunit.client.0.trial031.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-17T21:33:41.664 INFO:tasks.workunit.client.0.trial031.stdout: 0 0 0 0 0 0 - 0 2026-02-17T21:33:42.663 INFO:tasks.workunit.client.0.trial031.stdout: 1 15 10034 10019 39.134 39.1367 0.00198886 0.00159388 2026-02-17T21:33:43.663 INFO:tasks.workunit.client.0.trial031.stdout: 2 16 19795 19779 38.6281 38.125 0.00112606 0.00161598 2026-02-17T21:33:44.663 INFO:tasks.workunit.client.0.trial031.stdout: 3 16 29395 29379 38.2511 37.5 0.00149139 0.00163205 2026-02-17T21:33:45.663 INFO:tasks.workunit.client.0.trial031.stdout: 4 16 38883 38867 37.9533 37.0625 0.00149763 0.00164511 2026-02-17T21:33:46.663 INFO:tasks.workunit.client.0.trial031.stdout: 5 16 48424 48408 37.816 37.2695 0.00170318 0.00165109 2026-02-17T21:33:47.664 INFO:tasks.workunit.client.0.trial031.stdout: 6 16 57953 57937 37.7166 37.2227 0.00132747 0.00165541 2026-02-17T21:33:48.664 INFO:tasks.workunit.client.0.trial031.stdout: 7 16 67413 67397 37.6071 36.9531 0.0015808 0.00166033 2026-02-17T21:33:49.664 INFO:tasks.workunit.client.0.trial031.stdout: 8 16 76589 76573 37.3864 35.8438 0.00147636 0.0016702 2026-02-17T21:33:50.664 INFO:tasks.workunit.client.0.trial031.stdout: 9 15 85458 85443 37.0819 34.6484 0.00213988 0.00168391 2026-02-17T21:33:51.664 INFO:tasks.workunit.client.0.trial031.stdout: 10 16 94522 94506 36.9136 35.4023 0.00164661 0.0016916 2026-02-17T21:33:52.664 INFO:tasks.workunit.client.0.trial031.stdout: 11 16 103245 103229 36.6553 34.0742 0.00353164 0.00170353 2026-02-17T21:33:53.664 INFO:tasks.workunit.client.0.trial031.stdout: 12 16 110356 110340 35.9153 27.7773 0.0777629 0.00173486 2026-02-17T21:33:54.664 INFO:tasks.workunit.client.0.trial031.stdout: 13 16 114591 114575 34.4251 16.543 0.00143991 0.00181402 2026-02-17T21:33:55.664 INFO:tasks.workunit.client.0.trial031.stdout: 14 16 123546 123530 34.4646 34.9805 0.00208184 0.00181199 2026-02-17T21:33:56.664 INFO:tasks.workunit.client.0.trial031.stdout: 15 16 132552 132536 34.5121 35.1797 0.00185494 0.00180949 2026-02-17T21:33:57.665 INFO:tasks.workunit.client.0.trial031.stdout: 16 16 141560 141544 34.5541 35.1875 0.00235765 0.00180727 2026-02-17T21:33:58.664 INFO:tasks.workunit.client.0.trial031.stdout: 17 15 150700 150685 34.6218 35.707 0.00186938 0.00180378 2026-02-17T21:33:59.665 INFO:tasks.workunit.client.0.trial031.stdout: 18 16 157179 157163 34.1041 25.3047 0.0747406 0.00182808 2026-02-17T21:34:00.665 INFO:tasks.workunit.client.0.trial031.stdout: 19 16 161099 161083 33.115 15.3125 0.00180224 0.00188592 2026-02-17T21:34:01.665 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:34:01.668259+0000 min lat: 0.000586908 max lat: 0.137655 avg lat: 0.00188135 2026-02-17T21:34:01.665 INFO:tasks.workunit.client.0.trial031.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-17T21:34:01.665 INFO:tasks.workunit.client.0.trial031.stdout: 20 16 169984 169968 33.1945 34.707 0.00239157 0.00188135 2026-02-17T21:34:02.665 INFO:tasks.workunit.client.0.trial031.stdout: 21 16 178845 178829 33.2619 34.6133 0.00165419 0.00187759 2026-02-17T21:34:03.665 INFO:tasks.workunit.client.0.trial031.stdout: 22 16 188094 188078 33.3921 36.1289 0.00227512 0.00187027 2026-02-17T21:34:04.665 INFO:tasks.workunit.client.0.trial031.stdout: 23 16 197065 197049 33.4637 35.043 0.001293 0.0018662 2026-02-17T21:34:05.665 INFO:tasks.workunit.client.0.trial031.stdout: 24 16 202330 202314 32.9263 20.5664 0.0757588 0.00189169 2026-02-17T21:34:06.665 INFO:tasks.workunit.client.0.trial031.stdout: 25 16 202429 202413 31.6247 0.386719 0.0799342 0.00196375 2026-02-17T21:34:07.665 INFO:tasks.workunit.client.0.trial031.stdout: 26 16 206114 206098 30.962 14.3945 0.00171002 0.00201718 2026-02-17T21:34:08.665 INFO:tasks.workunit.client.0.trial031.stdout: 27 16 215080 215064 31.1123 35.0234 0.00266318 0.00200738 2026-02-17T21:34:09.665 INFO:tasks.workunit.client.0.trial031.stdout: 28 15 224500 224485 31.3153 36.8008 0.0016375 0.0019944 2026-02-17T21:34:10.666 INFO:tasks.workunit.client.0.trial031.stdout: 29 15 233616 233601 31.4633 35.6094 0.00170688 0.00198502 2026-02-17T21:34:11.666 INFO:tasks.workunit.client.0.trial031.stdout:Total time run: 30.0009 2026-02-17T21:34:11.666 INFO:tasks.workunit.client.0.trial031.stdout:Total writes made: 242950 2026-02-17T21:34:11.666 INFO:tasks.workunit.client.0.trial031.stdout:Write size: 4096 2026-02-17T21:34:11.666 INFO:tasks.workunit.client.0.trial031.stdout:Object size: 4096 2026-02-17T21:34:11.666 INFO:tasks.workunit.client.0.trial031.stdout:Bandwidth (MB/sec): 31.6332 2026-02-17T21:34:11.666 INFO:tasks.workunit.client.0.trial031.stdout:Stddev Bandwidth: 9.26205 2026-02-17T21:34:11.666 INFO:tasks.workunit.client.0.trial031.stdout:Max bandwidth (MB/sec): 39.1367 2026-02-17T21:34:11.666 INFO:tasks.workunit.client.0.trial031.stdout:Min bandwidth (MB/sec): 0.386719 2026-02-17T21:34:11.666 INFO:tasks.workunit.client.0.trial031.stdout:Average IOPS: 8098 2026-02-17T21:34:11.667 INFO:tasks.workunit.client.0.trial031.stdout:Stddev IOPS: 2371.09 2026-02-17T21:34:11.667 INFO:tasks.workunit.client.0.trial031.stdout:Max IOPS: 10019 2026-02-17T21:34:11.667 INFO:tasks.workunit.client.0.trial031.stdout:Min IOPS: 99 2026-02-17T21:34:11.667 INFO:tasks.workunit.client.0.trial031.stdout:Average Latency(s): 0.00197436 2026-02-17T21:34:11.667 INFO:tasks.workunit.client.0.trial031.stdout:Stddev Latency(s): 0.00539049 2026-02-17T21:34:11.667 INFO:tasks.workunit.client.0.trial031.stdout:Max latency(s): 0.260407 2026-02-17T21:34:11.667 INFO:tasks.workunit.client.0.trial031.stdout:Min latency(s): 0.000586908 2026-02-17T21:34:11.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: kill 118156 2026-02-17T21:34:11.670 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: sleep 1 2026-02-17T21:34:12.671 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: kill 118156 2026-02-17T21:34:12.671 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: sleep 1 2026-02-17T21:34:13.672 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: kill 118156 2026-02-17T21:34:13.672 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 234: kill: (118156) - No such process 2026-02-17T21:34:13.672 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:235: TEST_bluestore: ceph osd down 0 2026-02-17T21:34:14.227 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already down. 2026-02-17T21:34:14.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: kill 119301 2026-02-17T21:34:14.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: sleep 1 2026-02-17T21:34:15.241 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: kill 119301 2026-02-17T21:34:15.241 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: sleep 1 2026-02-17T21:34:16.242 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: kill 119301 2026-02-17T21:34:16.243 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 236: kill: (119301) - No such process 2026-02-17T21:34:16.243 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:237: TEST_bluestore: ceph osd down 1 2026-02-17T21:34:16.434 INFO:tasks.workunit.client.0.trial031.stderr:osd.1 is already down. 2026-02-17T21:34:16.443 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: kill 120421 2026-02-17T21:34:16.443 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: sleep 1 2026-02-17T21:34:17.445 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: kill 120421 2026-02-17T21:34:17.445 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 238: kill: (120421) - No such process 2026-02-17T21:34:17.445 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:239: TEST_bluestore: ceph osd down 2 2026-02-17T21:34:17.639 INFO:tasks.workunit.client.0.trial031.stderr:osd.2 is already down. 2026-02-17T21:34:17.648 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:240: TEST_bluestore: kill 121541 2026-02-17T21:34:17.648 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:240: TEST_bluestore: sleep 1 2026-02-17T21:34:18.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:240: TEST_bluestore: kill 121541 2026-02-17T21:34:18.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 240: kill: (121541) - No such process 2026-02-17T21:34:18.649 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:241: TEST_bluestore: ceph osd down 3 2026-02-17T21:34:18.896 INFO:tasks.workunit.client.0.trial031.stderr:osd.3 is already down. 2026-02-17T21:34:18.905 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:244: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-17T21:34:23.765 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:34:23.828 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:246: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/0/db2 count=1024 bs=1M 2026-02-17T21:34:24.453 INFO:tasks.workunit.client.0.trial031.stderr:1024+0 records in 2026-02-17T21:34:24.453 INFO:tasks.workunit.client.0.trial031.stderr:1024+0 records out 2026-02-17T21:34:24.453 INFO:tasks.workunit.client.0.trial031.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.623356 s, 1.7 GB/s 2026-02-17T21:34:24.453 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:247: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.db --dev-target td/osd-bluefs-volume-ops/0/db2 --command bluefs-bdev-migrate 2026-02-17T21:34:24.466 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:34:29.881 INFO:tasks.workunit.client.0.trial031.stdout: device removed:1 td/osd-bluefs-volume-ops/0/block.db 2026-02-17T21:34:29.881 INFO:tasks.workunit.client.0.trial031.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/0/db2 2026-02-17T21:34:29.907 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:252: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-17T21:34:34.513 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:34:34.573 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:256: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/0/wal2 count=512 bs=1M 2026-02-17T21:34:34.897 INFO:tasks.workunit.client.0.trial031.stderr:512+0 records in 2026-02-17T21:34:34.897 INFO:tasks.workunit.client.0.trial031.stderr:512+0 records out 2026-02-17T21:34:34.897 INFO:tasks.workunit.client.0.trial031.stderr:536870912 bytes (537 MB, 512 MiB) copied, 0.322705 s, 1.7 GB/s 2026-02-17T21:34:34.898 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:257: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.wal --dev-target td/osd-bluefs-volume-ops/0/wal2 --command bluefs-bdev-migrate 2026-02-17T21:34:34.913 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:34:39.825 INFO:tasks.workunit.client.0.trial031.stdout: device removed:0 td/osd-bluefs-volume-ops/0/block.wal 2026-02-17T21:34:39.825 INFO:tasks.workunit.client.0.trial031.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/0/wal2 2026-02-17T21:34:39.855 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:261: TEST_bluestore: rm -rf td/osd-bluefs-volume-ops/0/wal 2026-02-17T21:34:39.896 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:263: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-17T21:34:44.749 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:34:44.809 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:266: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-17T21:34:49.685 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:34:49.745 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:268: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/1/db2 count=1024 bs=1M 2026-02-17T21:34:50.345 INFO:tasks.workunit.client.0.trial031.stderr:1024+0 records in 2026-02-17T21:34:50.346 INFO:tasks.workunit.client.0.trial031.stderr:1024+0 records out 2026-02-17T21:34:50.346 INFO:tasks.workunit.client.0.trial031.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.59945 s, 1.8 GB/s 2026-02-17T21:34:50.346 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:269: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block.db --devs-source td/osd-bluefs-volume-ops/1/block.wal --dev-target td/osd-bluefs-volume-ops/1/db2 --command bluefs-bdev-migrate 2026-02-17T21:34:50.362 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:34:55.761 INFO:tasks.workunit.client.0.trial031.stdout: device removed:1 td/osd-bluefs-volume-ops/1/block.db 2026-02-17T21:34:55.761 INFO:tasks.workunit.client.0.trial031.stdout: device removed:0 td/osd-bluefs-volume-ops/1/block.wal 2026-02-17T21:34:55.761 INFO:tasks.workunit.client.0.trial031.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/1/db2 2026-02-17T21:34:55.791 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:275: TEST_bluestore: rm -rf td/osd-bluefs-volume-ops/1/db 2026-02-17T21:34:55.916 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:277: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-17T21:35:00.277 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:35:00.341 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:279: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block.db --dev-target td/osd-bluefs-volume-ops/1/block --command bluefs-bdev-migrate 2026-02-17T21:35:00.358 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:35:03.977 INFO:tasks.workunit.client.0.trial031.stdout: device removed:1 td/osd-bluefs-volume-ops/1/block.db 2026-02-17T21:35:04.010 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:284: TEST_bluestore: rm -rf td/osd-bluefs-volume-ops/1/db2 2026-02-17T21:35:04.137 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:286: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-17T21:35:07.997 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:35:08.061 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:289: TEST_bluestore: ceph-objectstore-tool --type bluestore --data-path td/osd-bluefs-volume-ops/2 --op fsck --no-mon-config 2026-02-17T21:35:12.641 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:35:12.705 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:292: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/2/db2 count=1024 bs=1M 2026-02-17T21:35:13.320 INFO:tasks.workunit.client.0.trial031.stderr:1024+0 records in 2026-02-17T21:35:13.320 INFO:tasks.workunit.client.0.trial031.stderr:1024+0 records out 2026-02-17T21:35:13.320 INFO:tasks.workunit.client.0.trial031.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.613516 s, 1.8 GB/s 2026-02-17T21:35:13.320 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:293: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block --dev-target td/osd-bluefs-volume-ops/2/db2 --command bluefs-bdev-migrate 2026-02-17T21:35:13.337 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:35:14.177 INFO:tasks.workunit.client.0.trial031.stderr:Unable to migrate Slow volume to new location, please allocate new DB or WAL with --bluefs-bdev-new-db(wal) command 2026-02-17T21:35:14.179 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:299: TEST_bluestore: '[' 1 -eq 0 ']' 2026-02-17T21:35:14.179 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:302: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-17T21:35:18.781 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:35:18.840 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:305: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/2/db2 count=1024 bs=1M 2026-02-17T21:35:20.176 INFO:tasks.workunit.client.0.trial031.stderr:1024+0 records in 2026-02-17T21:35:20.176 INFO:tasks.workunit.client.0.trial031.stderr:1024+0 records out 2026-02-17T21:35:20.176 INFO:tasks.workunit.client.0.trial031.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.18195 s, 908 MB/s 2026-02-17T21:35:20.176 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:307: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block --devs-source td/osd-bluefs-volume-ops/2/block.db --devs-source td/osd-bluefs-volume-ops/2/block.wal --dev-target td/osd-bluefs-volume-ops/2/db2 --command bluefs-bdev-migrate 2026-02-17T21:35:20.190 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:35:25.089 INFO:tasks.workunit.client.0.trial031.stdout: device removed:1 td/osd-bluefs-volume-ops/2/block.db 2026-02-17T21:35:25.089 INFO:tasks.workunit.client.0.trial031.stdout: device removed:0 td/osd-bluefs-volume-ops/2/block.wal 2026-02-17T21:35:25.089 INFO:tasks.workunit.client.0.trial031.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/2/db2 2026-02-17T21:35:25.119 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:314: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-17T21:35:29.477 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:35:29.536 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:317: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/3/wal2 count=1024 bs=1M 2026-02-17T21:35:30.153 INFO:tasks.workunit.client.0.trial031.stderr:1024+0 records in 2026-02-17T21:35:30.153 INFO:tasks.workunit.client.0.trial031.stderr:1024+0 records out 2026-02-17T21:35:30.153 INFO:tasks.workunit.client.0.trial031.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.615122 s, 1.7 GB/s 2026-02-17T21:35:30.153 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:319: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block --devs-source td/osd-bluefs-volume-ops/3/block.wal --dev-target td/osd-bluefs-volume-ops/3/wal2 --command bluefs-bdev-migrate 2026-02-17T21:35:30.166 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:35:34.801 INFO:tasks.workunit.client.0.trial031.stdout: device removed:1 td/osd-bluefs-volume-ops/3/block 2026-02-17T21:35:34.801 INFO:tasks.workunit.client.0.trial031.stdout: device removed:0 td/osd-bluefs-volume-ops/3/block.wal 2026-02-17T21:35:34.801 INFO:tasks.workunit.client.0.trial031.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/3/wal2 2026-02-17T21:35:34.829 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:325: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-17T21:35:39.185 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:35:39.247 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:327: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 0 2026-02-17T21:35:39.247 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:35:39.247 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:35:39.247 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T21:35:39.247 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:35:39.247 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-17T21:35:39.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:35:39.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:35:39.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:35:39.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:35:39.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-17T21:35:39.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-17T21:35:39.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:35:39.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:35:39.248 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:35:39.248 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:35:39.249 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:35:39.249 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:35:39.249 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:35:39.249 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:35:39.249 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:35:39.249 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:35:39.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:35:39.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:35:39.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:35:39.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:35:39.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:35:39.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:35:39.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:35:39.250 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:35:39.250 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:35:39.250 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:35:39.250 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-17T21:35:39.250 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:35:39.251 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T21:35:39.251 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:35:39.252 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-17T21:35:39.252 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T21:35:39.252 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:35:39.252 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:35:39.252 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:35:39.269 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:35:39.271+0000 7f9f304b98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:35:39.271 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:35:39.271+0000 7f9f304b98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:35:39.272 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:35:39.271+0000 7f9f304b98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:35:39.456 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T21:35:39.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:35:39.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:35:39.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:35:39.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:35:39.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:39.457 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:35:39.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:35:39.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:39.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:35:39.657 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:40.659 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:40.659 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:40.659 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:35:40.660 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:35:40.660 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:40.660 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:35:40.859 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:41.027 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:35:41.027+0000 7f9f304b98c0 -1 Falling back to public interface 2026-02-17T21:35:41.861 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:41.861 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:41.861 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:35:41.862 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:35:41.862 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:41.862 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:35:42.055 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:43.057 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:43.057 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:43.057 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:35:43.058 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:35:43.058 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:43.058 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:35:43.249 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:44.250 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:44.251 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:44.251 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:35:44.252 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:35:44.252 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:44.252 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:35:44.455 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:45.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:45.457 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:45.458 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:35:45.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:35:45.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:45.458 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:35:45.659 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:46.661 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:46.661 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:46.661 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:35:46.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:35:46.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:46.662 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:35:46.865 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:47.867 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:47.867 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:47.867 INFO:tasks.workunit.client.0.trial031.stdout:7 2026-02-17T21:35:47.868 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-17T21:35:47.868 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:47.868 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:35:48.060 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:49.062 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:49.062 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:49.062 INFO:tasks.workunit.client.0.trial031.stdout:8 2026-02-17T21:35:49.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-17T21:35:49.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:49.063 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:35:49.271 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:50.272 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:50.273 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:50.273 INFO:tasks.workunit.client.0.trial031.stdout:9 2026-02-17T21:35:50.274 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-17T21:35:50.274 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:50.274 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:35:50.492 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:51.189 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:35:51.187+0000 7f9f304b98c0 -1 osd.0 62 log_to_monitors true 2026-02-17T21:35:51.493 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:51.493 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:51.494 INFO:tasks.workunit.client.0.trial031.stdout:10 2026-02-17T21:35:51.494 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-17T21:35:51.494 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:51.495 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:35:51.691 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 72 up_thru 72 down_at 63 last_clean_interval [52,62) [v2:127.0.0.1:6802/4256160666,v1:127.0.0.1:6803/4256160666] [v2:127.0.0.1:6804/4256160666,v1:127.0.0.1:6805/4256160666] exists,up df635ca7-b5c9-471f-8ba7-205a3bf37cdb 2026-02-17T21:35:51.692 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:35:51.692 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:35:51.692 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:35:51.693 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:328: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-17T21:35:51.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:328: TEST_bluestore: osd_pid0=128842 2026-02-17T21:35:51.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:329: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 1 2026-02-17T21:35:51.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:35:51.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:35:51.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T21:35:51.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:35:51.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-17T21:35:51.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:35:51.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:35:51.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:35:51.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:35:51.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-17T21:35:51.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-17T21:35:51.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:35:51.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:35:51.696 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:35:51.696 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:35:51.696 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:35:51.696 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:35:51.696 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:35:51.697 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:35:51.697 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:35:51.698 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:35:51.698 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:35:51.698 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:35:51.698 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:35:51.698 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:35:51.698 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:35:51.698 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:35:51.698 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:35:51.698 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:35:51.698 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:35:51.699 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:35:51.699 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-17T21:35:51.699 INFO:tasks.workunit.client.0.trial031.stdout:start osd.1 2026-02-17T21:35:51.700 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T21:35:51.700 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:35:51.700 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/1/whoami 2026-02-17T21:35:51.700 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T21:35:51.700 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:35:51.700 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:35:51.700 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:35:51.714 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:35:51.715+0000 7f04c136c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:35:51.716 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:35:51.715+0000 7f04c136c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:35:51.716 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:35:51.715+0000 7f04c136c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:35:51.893 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T21:35:51.894 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:35:51.894 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T21:35:51.894 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:35:51.894 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:35:51.894 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:51.894 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:35:51.895 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:35:51.895 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:51.895 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:35:52.104 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:52.986 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:35:52.987+0000 7f04c136c8c0 -1 Falling back to public interface 2026-02-17T21:35:53.106 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:53.106 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:53.106 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:35:53.106 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:35:53.107 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:53.107 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:35:53.308 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:54.310 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:54.310 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:54.310 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:35:54.311 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:35:54.311 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:54.311 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:35:54.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:55.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:55.514 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:55.514 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:35:55.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:35:55.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:55.515 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:35:55.714 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:56.716 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:56.716 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:56.716 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:35:56.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:35:56.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:56.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:35:56.910 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:57.912 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:57.912 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:57.912 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:35:57.912 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:35:57.912 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:57.912 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:35:58.108 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:35:59.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:35:59.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:35:59.110 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:35:59.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:35:59.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:35:59.110 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:35:59.313 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:00.315 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:00.315 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:00.316 INFO:tasks.workunit.client.0.trial031.stdout:7 2026-02-17T21:36:00.316 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-17T21:36:00.316 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:00.317 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:36:00.519 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:01.520 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:01.520 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:01.521 INFO:tasks.workunit.client.0.trial031.stdout:8 2026-02-17T21:36:01.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-17T21:36:01.521 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:01.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:36:01.724 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:36:01.723+0000 7f04c136c8c0 -1 osd.1 64 log_to_monitors true 2026-02-17T21:36:01.731 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:02.732 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:02.732 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:02.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-17T21:36:02.733 INFO:tasks.workunit.client.0.trial031.stdout:9 2026-02-17T21:36:02.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:02.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T21:36:02.938 INFO:tasks.workunit.client.0.trial031.stdout:osd.1 up in weight 1 up_from 75 up_thru 75 down_at 65 last_clean_interval [55,64) [v2:127.0.0.1:6810/180995872,v1:127.0.0.1:6811/180995872] [v2:127.0.0.1:6812/180995872,v1:127.0.0.1:6813/180995872] exists,up 3060462c-e701-42c9-891d-76dd092c903e 2026-02-17T21:36:02.939 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:36:02.939 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:36:02.939 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:36:02.939 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:330: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-17T21:36:02.940 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:330: TEST_bluestore: osd_pid1=129987 2026-02-17T21:36:02.940 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:331: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 2 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:36:02.941 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:36:02.942 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:36:02.942 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:36:02.942 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:36:02.942 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:36:02.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:36:02.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:36:02.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:36:02.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:36:02.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:36:02.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:36:02.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:36:02.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:36:02.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:36:02.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:36:02.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-17T21:36:02.944 INFO:tasks.workunit.client.0.trial031.stdout:start osd.2 2026-02-17T21:36:02.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T21:36:02.944 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:36:02.945 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/2/whoami 2026-02-17T21:36:02.945 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T21:36:02.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:36:02.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:36:02.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:36:02.965 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:36:02.967+0000 7f86a4f5b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:36:02.967 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:36:02.967+0000 7f86a4f5b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:36:02.968 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:36:02.967+0000 7f86a4f5b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:36:03.150 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T21:36:03.150 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:36:03.150 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T21:36:03.150 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:36:03.151 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:36:03.151 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:03.151 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:36:03.152 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:36:03.152 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:03.152 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:36:03.349 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:04.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:04.351 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:04.351 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:36:04.352 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:36:04.352 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:04.352 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:36:04.546 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:04.707 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:36:04.707+0000 7f86a4f5b8c0 -1 Falling back to public interface 2026-02-17T21:36:05.547 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:05.548 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:05.548 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:36:05.549 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:36:05.549 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:05.549 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:36:05.743 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:06.745 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:06.745 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:06.745 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:36:06.745 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:36:06.746 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:06.746 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:36:06.948 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:07.950 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:07.950 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:07.951 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:36:07.951 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:36:07.951 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:07.952 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:36:08.155 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:09.156 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:09.157 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:09.157 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:36:09.158 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:36:09.158 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:09.158 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:36:09.359 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:10.360 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:10.360 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:10.361 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:36:10.361 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:36:10.361 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:10.361 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:36:10.563 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:11.565 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:11.565 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:11.565 INFO:tasks.workunit.client.0.trial031.stdout:7 2026-02-17T21:36:11.566 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-17T21:36:11.566 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:11.566 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:36:11.768 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:12.769 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:12.769 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:12.770 INFO:tasks.workunit.client.0.trial031.stdout:8 2026-02-17T21:36:12.770 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-17T21:36:12.771 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:12.771 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:36:12.973 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:13.801 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:36:13.802+0000 7f86a4f5b8c0 -1 osd.2 67 log_to_monitors true 2026-02-17T21:36:13.974 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:13.974 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:13.975 INFO:tasks.workunit.client.0.trial031.stdout:9 2026-02-17T21:36:13.975 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-17T21:36:13.976 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:13.976 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T21:36:14.179 INFO:tasks.workunit.client.0.trial031.stdout:osd.2 up in weight 1 up_from 78 up_thru 78 down_at 68 last_clean_interval [58,67) [v2:127.0.0.1:6818/2626434838,v1:127.0.0.1:6819/2626434838] [v2:127.0.0.1:6820/2626434838,v1:127.0.0.1:6821/2626434838] exists,up 3d0ce96d-e4ec-4623-a2b6-711f320be599 2026-02-17T21:36:14.181 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:36:14.181 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:36:14.181 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:36:14.181 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:332: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-17T21:36:14.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:332: TEST_bluestore: osd_pid2=131097 2026-02-17T21:36:14.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:333: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 3 2026-02-17T21:36:14.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:36:14.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:36:14.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T21:36:14.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:36:14.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-17T21:36:14.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-17T21:36:14.185 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:36:14.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:36:14.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:36:14.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-17T21:36:14.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-17T21:36:14.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:36:14.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:36:14.186 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:36:14.186 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:36:14.186 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:36:14.187 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:36:14.187 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:36:14.187 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:36:14.187 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:36:14.187 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:36:14.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:36:14.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:36:14.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:36:14.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:36:14.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:36:14.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:36:14.187 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:36:14.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:36:14.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:36:14.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:36:14.188 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-17T21:36:14.188 INFO:tasks.workunit.client.0.trial031.stdout:start osd.3 2026-02-17T21:36:14.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T21:36:14.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:36:14.189 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/3/whoami 2026-02-17T21:36:14.189 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T21:36:14.190 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:36:14.190 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:36:14.190 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:36:14.199 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:36:14.198+0000 7fcc83e068c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:36:14.200 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:36:14.198+0000 7fcc83e068c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:36:14.201 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:36:14.202+0000 7fcc83e068c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:36:14.384 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T21:36:14.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:36:14.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T21:36:14.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:36:14.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:36:14.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:14.386 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:36:14.386 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:36:14.386 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:14.386 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:36:14.588 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:15.590 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:15.590 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:15.590 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:36:15.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:36:15.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:15.591 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:36:15.792 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:15.906 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:36:15.906+0000 7fcc83e068c0 -1 Falling back to public interface 2026-02-17T21:36:16.793 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:16.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:16.794 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:36:16.794 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:36:16.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:16.795 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:36:16.996 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:17.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:17.998 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:17.998 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:36:17.999 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:36:17.999 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:17.999 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:36:18.196 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:19.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:19.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:19.198 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:36:19.199 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:36:19.199 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:19.199 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:36:19.405 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:20.407 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:20.407 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:20.408 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:36:20.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:36:20.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:20.409 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:36:20.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:21.615 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:21.615 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:21.616 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:36:21.616 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:36:21.617 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:21.617 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:36:21.822 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:22.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:22.824 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:22.824 INFO:tasks.workunit.client.0.trial031.stdout:7 2026-02-17T21:36:22.825 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-17T21:36:22.826 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:22.826 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:36:23.019 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:24.021 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:24.021 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:24.021 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-17T21:36:24.022 INFO:tasks.workunit.client.0.trial031.stdout:8 2026-02-17T21:36:24.022 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:24.023 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:36:24.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:25.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:25.219 INFO:tasks.workunit.client.0.trial031.stdout:9 2026-02-17T21:36:25.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:25.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-17T21:36:25.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:25.220 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:36:25.290 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:36:25.290+0000 7fcc83e068c0 -1 osd.3 69 log_to_monitors true 2026-02-17T21:36:25.425 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:36:26.427 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:36:26.427 INFO:tasks.workunit.client.0.trial031.stdout:10 2026-02-17T21:36:26.428 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:36:26.428 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-17T21:36:26.428 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:36:26.428 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T21:36:26.627 INFO:tasks.workunit.client.0.trial031.stdout:osd.3 up in weight 1 up_from 81 up_thru 81 down_at 70 last_clean_interval [61,69) [v2:127.0.0.1:6826/2646873886,v1:127.0.0.1:6827/2646873886] [v2:127.0.0.1:6828/2646873886,v1:127.0.0.1:6829/2646873886] exists,up 3c7e5245-677b-4faf-ac5c-593720b47b50 2026-02-17T21:36:26.628 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:36:26.629 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:36:26.629 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:36:26.629 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:334: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-17T21:36:26.629 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:334: TEST_bluestore: osd_pid3=132212 2026-02-17T21:36:26.629 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:337: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-17T21:36:26.646 INFO:tasks.workunit.client.0.trial031.stdout:hints = 1 2026-02-17T21:36:26.646 INFO:tasks.workunit.client.0.trial031.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-17T21:36:26.646 INFO:tasks.workunit.client.0.trial031.stdout:Object prefix: benchmark_data_trial031_133337 2026-02-17T21:36:26.646 INFO:tasks.workunit.client.0.trial031.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-17T21:36:26.646 INFO:tasks.workunit.client.0.trial031.stdout: 0 0 0 0 0 0 - 0 2026-02-17T21:36:27.646 INFO:tasks.workunit.client.0.trial031.stdout: 1 16 10406 10390 40.5826 40.5859 0.00143644 0.00153749 2026-02-17T21:36:28.646 INFO:tasks.workunit.client.0.trial031.stdout: 2 16 20728 20712 40.4499 40.3203 0.00459703 0.00154175 2026-02-17T21:36:29.646 INFO:tasks.workunit.client.0.trial031.stdout: 3 16 30939 30923 40.2611 39.8867 0.00162865 0.0015505 2026-02-17T21:36:30.646 INFO:tasks.workunit.client.0.trial031.stdout: 4 16 41284 41268 40.2976 40.4102 0.00109169 0.00154919 2026-02-17T21:36:31.646 INFO:tasks.workunit.client.0.trial031.stdout: 5 16 51436 51420 40.1687 39.6562 0.00153402 0.00155426 2026-02-17T21:36:32.646 INFO:tasks.workunit.client.0.trial031.stdout: 6 16 61368 61352 39.9395 38.7969 0.00278827 0.00156319 2026-02-17T21:36:33.646 INFO:tasks.workunit.client.0.trial031.stdout: 7 16 71225 71209 39.734 38.5039 0.00114277 0.00157125 2026-02-17T21:36:34.647 INFO:tasks.workunit.client.0.trial031.stdout: 8 16 75269 75253 36.7417 15.7969 0.00166353 0.00169943 2026-02-17T21:36:35.647 INFO:tasks.workunit.client.0.trial031.stdout: 9 16 84957 84941 36.8638 37.8438 0.00112212 0.00169387 2026-02-17T21:36:36.646 INFO:tasks.workunit.client.0.trial031.stdout: 10 15 94640 94625 36.96 37.8281 0.00166004 0.00168943 2026-02-17T21:36:37.647 INFO:tasks.workunit.client.0.trial031.stdout: 11 16 104486 104470 37.0958 38.457 0.00133101 0.00168331 2026-02-17T21:36:38.647 INFO:tasks.workunit.client.0.trial031.stdout: 12 16 114231 114215 37.1764 38.0664 0.0013427 0.00167963 2026-02-17T21:36:39.647 INFO:tasks.workunit.client.0.trial031.stdout: 13 15 123979 123964 37.2459 38.082 0.00175724 0.00167649 2026-02-17T21:36:40.647 INFO:tasks.workunit.client.0.trial031.stdout: 14 16 133635 133619 37.2791 37.7148 0.00143349 0.001675 2026-02-17T21:36:41.647 INFO:tasks.workunit.client.0.trial031.stdout: 15 16 143180 143164 37.2794 37.2852 0.00199468 0.00167502 2026-02-17T21:36:42.647 INFO:tasks.workunit.client.0.trial031.stdout: 16 16 152766 152750 37.2896 37.4453 0.00182616 0.00167453 2026-02-17T21:36:43.647 INFO:tasks.workunit.client.0.trial031.stdout: 17 16 162327 162311 37.2928 37.3477 0.00242466 0.0016744 2026-02-17T21:36:44.647 INFO:tasks.workunit.client.0.trial031.stdout: 18 16 171785 171769 37.2733 36.9453 0.00184881 0.00167531 2026-02-17T21:36:45.647 INFO:tasks.workunit.client.0.trial031.stdout: 19 16 172843 172827 35.5291 4.13281 0.0549858 0.00175177 2026-02-17T21:36:46.647 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:36:46.650744+0000 min lat: 0.000561152 max lat: 0.227456 avg lat: 0.00183632 2026-02-17T21:36:46.648 INFO:tasks.workunit.client.0.trial031.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-17T21:36:46.648 INFO:tasks.workunit.client.0.trial031.stdout: 20 16 172962 172946 33.7759 0.464844 0.1679 0.00183632 2026-02-17T21:36:47.647 INFO:tasks.workunit.client.0.trial031.stdout: 21 16 173295 173279 32.2294 1.30078 0.0016476 0.00193769 2026-02-17T21:36:48.648 INFO:tasks.workunit.client.0.trial031.stdout: 22 16 182774 182758 32.4474 37.0273 0.00115594 0.00192472 2026-02-17T21:36:49.648 INFO:tasks.workunit.client.0.trial031.stdout: 23 16 192405 192389 32.6722 37.6211 0.00112555 0.00191146 2026-02-17T21:36:50.648 INFO:tasks.workunit.client.0.trial031.stdout: 24 16 201878 201862 32.8525 37.0039 0.00100946 0.00190097 2026-02-17T21:36:51.648 INFO:tasks.workunit.client.0.trial031.stdout: 25 16 211457 211441 33.035 37.418 0.00168342 0.00189045 2026-02-17T21:36:52.648 INFO:tasks.workunit.client.0.trial031.stdout: 26 15 221019 221004 33.2011 37.3555 0.00122358 0.001881 2026-02-17T21:36:53.648 INFO:tasks.workunit.client.0.trial031.stdout: 27 16 230414 230398 33.3304 36.6953 0.00231876 0.00187371 2026-02-17T21:36:54.648 INFO:tasks.workunit.client.0.trial031.stdout: 28 16 239833 239817 33.454 36.793 0.00102342 0.00186677 2026-02-17T21:36:55.648 INFO:tasks.workunit.client.0.trial031.stdout: 29 16 249393 249377 33.588 37.3438 0.00200356 0.00185932 2026-02-17T21:36:56.648 INFO:tasks.workunit.client.0.trial031.stdout: 30 8 258667 258659 33.6769 36.2578 0.00121425 0.0018543 2026-02-17T21:36:57.648 INFO:tasks.workunit.client.0.trial031.stdout:Total time run: 30.0038 2026-02-17T21:36:57.649 INFO:tasks.workunit.client.0.trial031.stdout:Total writes made: 258667 2026-02-17T21:36:57.649 INFO:tasks.workunit.client.0.trial031.stdout:Write size: 4096 2026-02-17T21:36:57.649 INFO:tasks.workunit.client.0.trial031.stdout:Object size: 4096 2026-02-17T21:36:57.649 INFO:tasks.workunit.client.0.trial031.stdout:Bandwidth (MB/sec): 33.6763 2026-02-17T21:36:57.649 INFO:tasks.workunit.client.0.trial031.stdout:Stddev Bandwidth: 11.5564 2026-02-17T21:36:57.649 INFO:tasks.workunit.client.0.trial031.stdout:Max bandwidth (MB/sec): 40.5859 2026-02-17T21:36:57.649 INFO:tasks.workunit.client.0.trial031.stdout:Min bandwidth (MB/sec): 0.464844 2026-02-17T21:36:57.649 INFO:tasks.workunit.client.0.trial031.stdout:Average IOPS: 8621 2026-02-17T21:36:57.649 INFO:tasks.workunit.client.0.trial031.stdout:Stddev IOPS: 2958.43 2026-02-17T21:36:57.649 INFO:tasks.workunit.client.0.trial031.stdout:Max IOPS: 10390 2026-02-17T21:36:57.649 INFO:tasks.workunit.client.0.trial031.stdout:Min IOPS: 119 2026-02-17T21:36:57.649 INFO:tasks.workunit.client.0.trial031.stdout:Average Latency(s): 0.00185438 2026-02-17T21:36:57.650 INFO:tasks.workunit.client.0.trial031.stdout:Stddev Latency(s): 0.00577949 2026-02-17T21:36:57.650 INFO:tasks.workunit.client.0.trial031.stdout:Max latency(s): 0.359411 2026-02-17T21:36:57.650 INFO:tasks.workunit.client.0.trial031.stdout:Min latency(s): 0.000561152 2026-02-17T21:36:57.655 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:339: TEST_bluestore: wait_for_clean 2026-02-17T21:36:57.655 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:36:57.655 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:36:57.656 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:36:57.656 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T21:36:57.657 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:36:57.657 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:36:57.657 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:36:57.657 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:36:57.657 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:36:57.757 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T21:36:57.757 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:36:57.757 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:36:57.757 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:36:57.758 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:36:57.758 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:36:58.488 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T21:36:58.488 INFO:tasks.workunit.client.0.trial031.stderr:1 2026-02-17T21:36:58.488 INFO:tasks.workunit.client.0.trial031.stderr:2 2026-02-17T21:36:58.488 INFO:tasks.workunit.client.0.trial031.stderr:3' 2026-02-17T21:36:58.488 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:36:58.488 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:36:58.489 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:36:58.704 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=309237645328 2026-02-17T21:36:58.704 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 309237645328 2026-02-17T21:36:58.704 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-309237645328' 2026-02-17T21:36:58.704 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:36:58.705 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T21:36:59.275 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=322122547214 2026-02-17T21:36:59.275 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 322122547214 2026-02-17T21:36:59.275 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-309237645328 1-322122547214' 2026-02-17T21:36:59.275 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:36:59.275 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T21:36:59.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=335007449100 2026-02-17T21:36:59.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 335007449100 2026-02-17T21:36:59.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-309237645328 1-322122547214 2-335007449100' 2026-02-17T21:36:59.350 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:36:59.350 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T21:36:59.497 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=347892350985 2026-02-17T21:36:59.497 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 347892350985 2026-02-17T21:36:59.497 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-309237645328 1-322122547214 2-335007449100 3-347892350985' 2026-02-17T21:36:59.497 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:36:59.498 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-309237645328 2026-02-17T21:36:59.498 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:36:59.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:36:59.500 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-309237645328 2026-02-17T21:36:59.501 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:36:59.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=309237645328 2026-02-17T21:36:59.502 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 309237645328 2026-02-17T21:36:59.502 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 309237645328' 2026-02-17T21:36:59.503 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:36:59.979 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 309237645328 -lt 309237645328 2026-02-17T21:36:59.979 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:36:59.980 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-322122547214 2026-02-17T21:36:59.980 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:36:59.982 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T21:36:59.983 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-322122547214 2026-02-17T21:36:59.983 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:36:59.984 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=322122547214 2026-02-17T21:36:59.984 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 322122547214' 2026-02-17T21:36:59.985 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.1 seq 322122547214 2026-02-17T21:36:59.985 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:37:00.773 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 322122547212 -lt 322122547214 2026-02-17T21:37:00.774 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:37:01.775 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:37:01.775 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T21:37:01.976 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 322122547214 -lt 322122547214 2026-02-17T21:37:01.976 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:37:01.976 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-335007449100 2026-02-17T21:37:01.977 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:37:01.978 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T21:37:01.979 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-335007449100 2026-02-17T21:37:01.979 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:37:01.980 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=335007449100 2026-02-17T21:37:01.980 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.2 seq 335007449100 2026-02-17T21:37:01.981 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 335007449100' 2026-02-17T21:37:01.981 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T21:37:02.175 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 335007449100 -lt 335007449100 2026-02-17T21:37:02.175 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:37:02.176 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-347892350985 2026-02-17T21:37:02.176 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:37:02.177 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T21:37:02.178 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-347892350985 2026-02-17T21:37:02.178 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:37:02.179 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=347892350985 2026-02-17T21:37:02.180 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.3 seq 347892350985 2026-02-17T21:37:02.180 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 347892350985' 2026-02-17T21:37:02.180 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T21:37:02.389 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 347892350986 -lt 347892350985 2026-02-17T21:37:02.389 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:37:02.390 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:37:02.390 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:37:02.658 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 == 0 2026-02-17T21:37:02.658 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:37:02.658 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:37:02.659 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:37:02.659 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:37:02.659 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:37:02.659 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:37:02.659 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:37:02.864 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=16 2026-02-17T21:37:02.865 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:37:02.866 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:37:02.866 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:37:03.124 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 16 = 16 2026-02-17T21:37:03.124 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T21:37:03.124 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T21:37:03.124 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:15: run: teardown td/osd-bluefs-volume-ops 2026-02-17T21:37:03.124 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:37:03.124 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:37:03.125 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-17T21:37:03.125 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:37:03.125 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:37:03.125 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:37:03.125 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:37:03.125 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:37:03.434 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:37:03.434 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:37:03.435 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:37:03.435 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:37:03.436 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:37:03.436 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:37:03.437 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:37:03.438 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:37:03.438 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:37:03.438 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:37:03.438 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:37:03.439 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:37:03.441 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:37:03.441 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-17T21:37:09.195 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:37:09.195 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:37:09.196 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:37:09.196 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86056 2026-02-17T21:37:09.197 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:37:09.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:37:09.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:12: run: for func in $funcs 2026-02-17T21:37:09.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:13: run: setup td/osd-bluefs-volume-ops 2026-02-17T21:37:09.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:37:09.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-bluefs-volume-ops 2026-02-17T21:37:09.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:37:09.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:37:09.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-17T21:37:09.198 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:37:09.198 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:37:09.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:37:09.198 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:37:09.199 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:37:09.200 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:37:09.200 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:37:09.200 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:37:09.201 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:37:09.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:37:09.202 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:37:09.203 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:37:09.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:37:09.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:37:09.204 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:37:09.205 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:37:09.206 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:37:09.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:37:09.207 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-17T21:37:09.208 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:37:09.208 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:37:09.208 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:37:09.208 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86056 2026-02-17T21:37:09.209 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:37:09.210 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:37:09.210 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-bluefs-volume-ops 2026-02-17T21:37:09.211 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T21:37:09.211 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:37:09.211 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:37:09.211 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.86056 2026-02-17T21:37:09.213 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T21:37:09.214 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T21:37:09.214 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T21:37:09.214 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-bluefs-volume-ops 1' TERM HUP INT 2026-02-17T21:37:09.214 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: TEST_bluestore2 td/osd-bluefs-volume-ops 2026-02-17T21:37:09.214 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:343: TEST_bluestore2: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:37:09.214 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:345: TEST_bluestore2: ulimit -n 2026-02-17T21:37:09.215 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:345: TEST_bluestore2: local flimit=4096 2026-02-17T21:37:09.215 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:346: TEST_bluestore2: '[' 4096 -lt 1536 ']' 2026-02-17T21:37:09.215 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:349: TEST_bluestore2: export CEPH_MON=127.0.0.1:7146 2026-02-17T21:37:09.215 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:349: TEST_bluestore2: CEPH_MON=127.0.0.1:7146 2026-02-17T21:37:09.215 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:350: TEST_bluestore2: export CEPH_ARGS 2026-02-17T21:37:09.215 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:351: TEST_bluestore2: uuidgen 2026-02-17T21:37:09.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:351: TEST_bluestore2: CEPH_ARGS+='--fsid=9cb0c1bf-8dae-4aee-9568-2890d09706c5 --auth-supported=none ' 2026-02-17T21:37:09.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:352: TEST_bluestore2: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-17T21:37:09.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:353: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_size=4294967296 ' 2026-02-17T21:37:09.217 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:354: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_db_create=true ' 2026-02-17T21:37:09.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:355: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_db_size=1073741824 ' 2026-02-17T21:37:09.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:356: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_wal_create=false ' 2026-02-17T21:37:09.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:357: TEST_bluestore2: CEPH_ARGS+='--bluestore_fsck_on_mount=true ' 2026-02-17T21:37:09.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:358: TEST_bluestore2: CEPH_ARGS+='--osd_pool_default_size=1 ' 2026-02-17T21:37:09.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:359: TEST_bluestore2: CEPH_ARGS+='--osd_pool_default_min_size=1 ' 2026-02-17T21:37:09.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:360: TEST_bluestore2: CEPH_ARGS+='--bluestore_debug_enforce_settings=ssd ' 2026-02-17T21:37:09.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:362: TEST_bluestore2: run_mon td/osd-bluefs-volume-ops a 2026-02-17T21:37:09.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:37:09.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T21:37:09.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T21:37:09.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T21:37:09.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-bluefs-volume-ops/a 2026-02-17T21:37:09.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bluefs-volume-ops/a --run-dir=td/osd-bluefs-volume-ops 2026-02-17T21:37:09.239 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T21:37:09.239 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:37:09.239 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:37:09.240 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:37:09.240 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:37:09.240 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:37:09.240 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:37:09.240 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bluefs-volume-ops/a '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bluefs-volume-ops/log --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T21:37:09.267 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T21:37:09.268 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T21:37:09.268 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:37:09.268 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:37:09.268 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T21:37:09.268 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T21:37:09.269 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:37:09.269 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:37:09.269 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:37:09.269 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:37:09.269 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:37:09.270 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:37:09.270 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86056/ceph-mon.a.asok 2026-02-17T21:37:09.270 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:37:09.270 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.86056/ceph-mon.a.asok config get fsid 2026-02-17T21:37:09.327 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T21:37:09.327 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:37:09.327 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:37:09.327 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T21:37:09.327 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T21:37:09.328 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:37:09.328 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:37:09.328 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:37:09.329 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:37:09.329 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:37:09.329 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:37:09.329 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86056/ceph-mon.a.asok 2026-02-17T21:37:09.329 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:37:09.329 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.86056/ceph-mon.a.asok config get mon_host 2026-02-17T21:37:09.384 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:363: TEST_bluestore2: run_mgr td/osd-bluefs-volume-ops x 2026-02-17T21:37:09.384 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:37:09.384 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T21:37:09.384 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T21:37:09.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T21:37:09.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-bluefs-volume-ops/x 2026-02-17T21:37:09.385 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T21:37:09.498 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T21:37:09.498 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:37:09.498 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:37:09.499 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:37:09.499 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:37:09.499 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:37:09.499 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:37:09.499 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:37:09.500 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bluefs-volume-ops/x '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:37:09.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:364: TEST_bluestore2: run_osd td/osd-bluefs-volume-ops 0 2026-02-17T21:37:09.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:37:09.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:37:09.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T21:37:09.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:37:09.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-17T21:37:09.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=9cb0c1bf-8dae-4aee-9568-2890d09706c5 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-17T21:37:09.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:37:09.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:37:09.523 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:37:09.523 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-17T21:37:09.523 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-17T21:37:09.523 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:37:09.523 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:37:09.523 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:37:09.523 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:37:09.523 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:37:09.523 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:37:09.523 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:37:09.523 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:37:09.523 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:37:09.523 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:37:09.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:37:09.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:37:09.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:37:09.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:37:09.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:37:09.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:37:09.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:37:09.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:37:09.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:37:09.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:37:09.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:37:09.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:37:09.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-17T21:37:09.525 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:37:09.525 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b410d43f-ea79-432a-89ac-76c72b1e8d6c 2026-02-17T21:37:09.526 INFO:tasks.workunit.client.0.trial031.stdout:add osd0 b410d43f-ea79-432a-89ac-76c72b1e8d6c 2026-02-17T21:37:09.526 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 b410d43f-ea79-432a-89ac-76c72b1e8d6c' 2026-02-17T21:37:09.526 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:37:09.535 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCF35RpTXQOIBAAQPkCXlFsnI8XUVTxNgbyLA== 2026-02-17T21:37:09.536 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCF35RpTXQOIBAAQPkCXlFsnI8XUVTxNgbyLA=="}' 2026-02-17T21:37:09.536 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b410d43f-ea79-432a-89ac-76c72b1e8d6c -i td/osd-bluefs-volume-ops/0/new.json 2026-02-17T21:37:09.645 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:37:09.652 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/0/new.json 2026-02-17T21:37:09.653 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=9cb0c1bf-8dae-4aee-9568-2890d09706c5 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCF35RpTXQOIBAAQPkCXlFsnI8XUVTxNgbyLA== --osd-uuid b410d43f-ea79-432a-89ac-76c72b1e8d6c 2026-02-17T21:37:09.671 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:37:09.669+0000 7f435d1e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:37:09.672 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:37:09.669+0000 7f435d1e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:37:09.673 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:37:09.673+0000 7f435d1e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:37:09.674 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:37:09.673+0000 7f435d1e08c0 -1 bdev(0x55813a4b2c00 td/osd-bluefs-volume-ops/0/block) open stat got: (1) Operation not permitted 2026-02-17T21:37:09.674 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:37:09.673+0000 7f435d1e08c0 -1 bluestore(td/osd-bluefs-volume-ops/0) _read_fsid unparsable uuid 2026-02-17T21:37:14.749 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/0/keyring 2026-02-17T21:37:14.749 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:37:14.749 INFO:tasks.workunit.client.0.trial031.stdout:adding osd0 key to auth repository 2026-02-17T21:37:14.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T21:37:14.750 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:37:15.034 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:37:15.035 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T21:37:15.035 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=9cb0c1bf-8dae-4aee-9568-2890d09706c5 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:37:15.035 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:37:15.035 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:37:15.035 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:37:15.060 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:37:15.056+0000 7f1ec713c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:37:15.062 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:37:15.060+0000 7f1ec713c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:37:15.063 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:37:15.060+0000 7f1ec713c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:37:15.231 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T21:37:15.231 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:37:15.231 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:37:15.231 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:37:15.231 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:37:15.232 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:37:15.232 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:37:15.232 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:37:15.232 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:37:15.233 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:37:15.437 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:37:16.438 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:37:16.438 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:37:16.438 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:37:16.439 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:37:16.439 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:37:16.439 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:37:16.642 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:37:16.802 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:37:16.800+0000 7f1ec713c8c0 -1 Falling back to public interface 2026-02-17T21:37:17.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:37:17.644 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:37:17.644 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:37:17.645 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:37:17.645 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:37:17.645 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:37:17.844 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:37:18.846 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:37:18.846 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:37:18.846 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:37:18.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:37:18.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:37:18.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:37:19.054 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:37:20.056 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:37:20.056 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:37:20.056 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:37:20.057 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:37:20.057 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:37:20.057 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:37:20.264 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:37:21.266 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:37:21.266 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:37:21.266 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:37:21.267 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:37:21.267 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:37:21.267 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:37:21.407 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:37:21.404+0000 7f1ec713c8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T21:37:21.473 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:37:22.475 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:37:22.475 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:37:22.475 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:37:22.476 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:37:22.476 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:37:22.476 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:37:22.683 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2843715524,v1:127.0.0.1:6803/2843715524] [v2:127.0.0.1:6804/2843715524,v1:127.0.0.1:6805/2843715524] exists,up b410d43f-ea79-432a-89ac-76c72b1e8d6c 2026-02-17T21:37:22.684 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:37:22.684 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:37:22.684 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:37:22.684 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:365: TEST_bluestore2: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-17T21:37:22.685 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:365: TEST_bluestore2: osd_pid0=135255 2026-02-17T21:37:22.685 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:367: TEST_bluestore2: sleep 5 2026-02-17T21:37:27.687 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:368: TEST_bluestore2: create_pool foo 16 2026-02-17T21:37:27.687 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 16 2026-02-17T21:37:27.957 INFO:tasks.workunit.client.0.trial031.stderr:pool 'foo' already exists 2026-02-17T21:37:27.967 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T21:37:28.968 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:370: TEST_bluestore2: retry = 0 2026-02-17T21:37:28.969 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 370: retry: command not found 2026-02-17T21:37:28.969 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:371: TEST_bluestore2: [[ '' -le 5 ]] 2026-02-17T21:37:28.969 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:373: TEST_bluestore2: timeout 60 rados bench -p foo 10 write --write-omap --no-cleanup 2026-02-17T21:37:28.991 INFO:tasks.workunit.client.0.trial031.stdout:hints = 1 2026-02-17T21:37:28.992 INFO:tasks.workunit.client.0.trial031.stdout:Maintaining 16 concurrent writes of 4194304 bytes to objects of size 4194304 for up to 10 seconds or 0 objects 2026-02-17T21:37:28.992 INFO:tasks.workunit.client.0.trial031.stdout:Object prefix: benchmark_data_trial031_137743 2026-02-17T21:37:29.038 INFO:tasks.workunit.client.0.trial031.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-17T21:37:29.038 INFO:tasks.workunit.client.0.trial031.stdout: 0 0 0 0 0 0 - 0 2026-02-17T21:37:30.039 INFO:tasks.workunit.client.0.trial031.stdout: 1 16 180 164 655.945 656 0.0883779 0.0913096 2026-02-17T21:37:31.039 INFO:tasks.workunit.client.0.trial031.stdout: 2 16 382 366 731.938 808 0.0751521 0.086052 2026-02-17T21:37:32.039 INFO:tasks.workunit.client.0.trial031.stdout: 3 16 590 574 765.266 832 0.0815873 0.0830657 2026-02-17T21:37:33.039 INFO:tasks.workunit.client.0.trial031.stdout: 4 16 782 766 765.932 768 0.0784548 0.0826075 2026-02-17T21:37:34.039 INFO:tasks.workunit.client.0.trial031.stdout: 5 16 1006 990 791.93 896 0.0700674 0.0803336 2026-02-17T21:37:35.039 INFO:tasks.workunit.client.0.trial031.stdout: 6 16 1204 1188 791.929 792 0.0735203 0.0801164 2026-02-17T21:37:36.039 INFO:tasks.workunit.client.0.trial031.stdout: 7 16 1396 1380 788.501 768 0.0747155 0.080546 2026-02-17T21:37:37.039 INFO:tasks.workunit.client.0.trial031.stdout: 8 16 1598 1582 790.93 808 0.071246 0.0805325 2026-02-17T21:37:38.039 INFO:tasks.workunit.client.0.trial031.stdout: 9 16 1796 1780 791.041 792 0.0922816 0.0805031 2026-02-17T21:37:39.040 INFO:tasks.workunit.client.0.trial031.stdout: 10 16 1972 1956 782.329 704 0.0992479 0.0811184 2026-02-17T21:37:40.040 INFO:tasks.workunit.client.0.trial031.stdout:Total time run: 10.039 2026-02-17T21:37:40.040 INFO:tasks.workunit.client.0.trial031.stdout:Total writes made: 1972 2026-02-17T21:37:40.040 INFO:tasks.workunit.client.0.trial031.stdout:Write size: 4194304 2026-02-17T21:37:40.040 INFO:tasks.workunit.client.0.trial031.stdout:Object size: 4194304 2026-02-17T21:37:40.040 INFO:tasks.workunit.client.0.trial031.stdout:Bandwidth (MB/sec): 785.738 2026-02-17T21:37:40.040 INFO:tasks.workunit.client.0.trial031.stdout:Stddev Bandwidth: 66.1634 2026-02-17T21:37:40.040 INFO:tasks.workunit.client.0.trial031.stdout:Max bandwidth (MB/sec): 896 2026-02-17T21:37:40.041 INFO:tasks.workunit.client.0.trial031.stdout:Min bandwidth (MB/sec): 656 2026-02-17T21:37:40.041 INFO:tasks.workunit.client.0.trial031.stdout:Average IOPS: 196 2026-02-17T21:37:40.041 INFO:tasks.workunit.client.0.trial031.stdout:Stddev IOPS: 16.5409 2026-02-17T21:37:40.041 INFO:tasks.workunit.client.0.trial031.stdout:Max IOPS: 224 2026-02-17T21:37:40.041 INFO:tasks.workunit.client.0.trial031.stdout:Min IOPS: 164 2026-02-17T21:37:40.041 INFO:tasks.workunit.client.0.trial031.stdout:Average Latency(s): 0.0812815 2026-02-17T21:37:40.041 INFO:tasks.workunit.client.0.trial031.stdout:Stddev Latency(s): 0.0101388 2026-02-17T21:37:40.041 INFO:tasks.workunit.client.0.trial031.stdout:Max latency(s): 0.139546 2026-02-17T21:37:40.041 INFO:tasks.workunit.client.0.trial031.stdout:Min latency(s): 0.0373621 2026-02-17T21:37:40.050 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:376: TEST_bluestore2: sleep 10 2026-02-17T21:37:50.052 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:378: TEST_bluestore2: ceph tell osd.0 perf dump bluefs 2026-02-17T21:37:50.053 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:378: TEST_bluestore2: jq .bluefs.db_used_bytes 2026-02-17T21:37:50.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:378: TEST_bluestore2: db_used=793772032 2026-02-17T21:37:50.135 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:379: TEST_bluestore2: ceph tell osd.0 perf dump bluefs 2026-02-17T21:37:50.135 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:379: TEST_bluestore2: jq .bluefs.slow_used_bytes 2026-02-17T21:37:50.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:379: TEST_bluestore2: spilled_over=105840640 2026-02-17T21:37:50.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:380: TEST_bluestore2: (( retry+=1 )) 2026-02-17T21:37:50.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:381: TEST_bluestore2: test 105840640 -eq 0 2026-02-17T21:37:50.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:381: TEST_bluestore2: break 2026-02-17T21:37:50.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:383: TEST_bluestore2: test 105840640 -gt 0 2026-02-17T21:37:50.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:385: TEST_bluestore2: kill 135255 2026-02-17T21:37:50.218 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:385: TEST_bluestore2: sleep 1 2026-02-17T21:37:51.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:385: TEST_bluestore2: kill 135255 2026-02-17T21:37:51.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 385: kill: (135255) - No such process 2026-02-17T21:37:51.219 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:386: TEST_bluestore2: ceph osd down 0 2026-02-17T21:37:51.416 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already down. 2026-02-17T21:37:51.425 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:388: TEST_bluestore2: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.db --dev-target td/osd-bluefs-volume-ops/0/block --command bluefs-bdev-migrate 2026-02-17T21:37:51.444 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:37:54.837 INFO:tasks.workunit.client.0.trial031.stdout: device removed:1 td/osd-bluefs-volume-ops/0/block.db 2026-02-17T21:37:54.871 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:393: TEST_bluestore2: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --command bluefs-bdev-sizes 2026-02-17T21:37:54.892 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:37:55.979 INFO:tasks.workunit.client.0.trial031.stdout:1 : device size 0x100000000(4 GiB) : using 0x34408000(836 MiB) 2026-02-17T21:37:56.474 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:396: TEST_bluestore2: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --command fsck 2026-02-17T21:37:58.849 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:37:58.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:399: TEST_bluestore2: activate_osd td/osd-bluefs-volume-ops 0 2026-02-17T21:37:58.925 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:37:58.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:37:58.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T21:37:58.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:37:58.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-17T21:37:58.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=9cb0c1bf-8dae-4aee-9568-2890d09706c5 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-17T21:37:58.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:37:58.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:37:58.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:37:58.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-17T21:37:58.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-17T21:37:58.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:37:58.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:37:58.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:37:58.927 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:37:58.927 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:37:58.927 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:37:58.927 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:37:58.927 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:37:58.927 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:37:58.927 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:37:58.928 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:37:58.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:37:58.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:37:58.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:37:58.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:37:58.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:37:58.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:37:58.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:37:58.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:37:58.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:37:58.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-17T21:37:58.930 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:37:58.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T21:37:58.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=9cb0c1bf-8dae-4aee-9568-2890d09706c5 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:37:58.931 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-17T21:37:58.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T21:37:58.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:37:58.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:37:58.931 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:37:58.954 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:37:58.955+0000 7f07a68398c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:37:58.955 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:37:58.955+0000 7f07a68398c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:37:58.956 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:37:58.955+0000 7f07a68398c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:37:59.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T21:37:59.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:37:59.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:37:59.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:37:59.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:37:59.134 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:37:59.135 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:37:59.135 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:37:59.135 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:37:59.135 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:37:59.333 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:38:00.199 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:38:00.199+0000 7f07a68398c0 -1 Falling back to public interface 2026-02-17T21:38:00.334 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:38:00.335 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:38:00.335 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:38:00.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:38:00.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:38:00.336 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:38:00.534 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:38:01.535 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:38:01.536 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:38:01.536 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:38:01.537 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:38:01.537 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:38:01.537 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:38:01.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:38:02.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:38:02.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:38:02.736 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:38:02.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:38:02.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:38:02.737 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:38:02.943 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:38:03.945 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:38:03.945 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:38:03.945 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:38:03.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:38:03.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:38:03.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:38:04.160 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:38:05.161 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:38:05.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:38:05.162 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:38:05.163 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:38:05.163 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:38:05.163 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:38:05.366 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:38:05.703 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:38:05.702+0000 7f07a68398c0 -1 osd.0 9 log_to_monitors true 2026-02-17T21:38:06.368 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:38:06.368 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:38:06.368 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:38:06.369 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:38:06.369 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:38:06.369 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:38:06.566 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 12 up_thru 12 down_at 10 last_clean_interval [5,9) [v2:127.0.0.1:6802/3902650516,v1:127.0.0.1:6803/3902650516] [v2:127.0.0.1:6804/3902650516,v1:127.0.0.1:6805/3902650516] exists,up b410d43f-ea79-432a-89ac-76c72b1e8d6c 2026-02-17T21:38:06.567 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:38:06.567 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:38:06.567 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:38:06.567 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:400: TEST_bluestore2: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-17T21:38:06.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:400: TEST_bluestore2: osd_pid0=138974 2026-02-17T21:38:06.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:402: TEST_bluestore2: wait_for_clean 2026-02-17T21:38:06.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:38:06.568 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:38:06.569 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:38:06.569 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T21:38:06.569 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:38:06.570 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:38:06.570 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:38:06.570 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:38:06.570 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:38:06.661 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T21:38:06.661 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:38:06.661 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:38:06.661 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:38:06.661 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:38:06.661 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:38:06.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T21:38:06.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:38:06.877 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:38:06.877 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:38:06.954 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607555 2026-02-17T21:38:06.954 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607555 2026-02-17T21:38:06.954 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-51539607555' 2026-02-17T21:38:06.955 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:38:06.955 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-51539607555 2026-02-17T21:38:06.955 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:38:06.956 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:38:06.957 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-51539607555 2026-02-17T21:38:06.957 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:38:06.958 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607555 2026-02-17T21:38:06.958 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 51539607555 2026-02-17T21:38:06.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 51539607555' 2026-02-17T21:38:06.959 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:38:07.167 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 51539607555 2026-02-17T21:38:07.167 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:38:08.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:38:08.169 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:38:08.377 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607555 -lt 51539607555 2026-02-17T21:38:08.377 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:38:08.378 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:38:08.378 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:38:08.642 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 == 0 2026-02-17T21:38:08.642 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:38:08.643 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:38:08.643 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:38:08.643 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:38:08.643 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:38:08.643 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:38:08.643 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:38:08.847 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=16 2026-02-17T21:38:08.848 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:38:08.848 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:38:08.848 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:38:09.119 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 16 = 16 2026-02-17T21:38:09.119 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T21:38:09.120 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T21:38:09.120 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:15: run: teardown td/osd-bluefs-volume-ops 2026-02-17T21:38:09.120 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:38:09.120 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:38:09.120 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-17T21:38:09.120 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:38:09.120 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:38:09.120 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:38:09.120 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:38:09.120 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:38:09.227 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:38:09.228 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:38:09.228 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:38:09.228 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:38:09.229 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:38:09.229 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:38:09.230 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:38:09.231 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:38:09.231 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:38:09.231 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:38:09.231 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:38:09.232 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:38:09.234 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:38:09.234 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-17T21:38:09.391 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:38:09.391 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:38:09.392 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:38:09.392 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86056 2026-02-17T21:38:09.394 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:38:09.394 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:38:09.394 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:12: run: for func in $funcs 2026-02-17T21:38:09.394 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:13: run: setup td/osd-bluefs-volume-ops 2026-02-17T21:38:09.394 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:38:09.394 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-bluefs-volume-ops 2026-02-17T21:38:09.394 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:38:09.394 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T21:38:09.394 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-17T21:38:09.394 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:38:09.394 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:38:09.395 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:38:09.395 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:38:09.395 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:38:09.396 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:38:09.397 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:38:09.397 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:38:09.398 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:38:09.399 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:38:09.399 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:38:09.400 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:38:09.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:38:09.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:38:09.401 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:38:09.402 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:38:09.403 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:38:09.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T21:38:09.404 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-17T21:38:09.405 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:38:09.405 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:38:09.405 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:38:09.405 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86056 2026-02-17T21:38:09.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:38:09.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:38:09.406 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-bluefs-volume-ops 2026-02-17T21:38:09.408 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T21:38:09.408 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:38:09.408 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:38:09.408 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.86056 2026-02-17T21:38:09.410 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T21:38:09.410 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T21:38:09.411 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T21:38:09.411 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-bluefs-volume-ops 1' TERM HUP INT 2026-02-17T21:38:09.411 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: TEST_bluestore_expand td/osd-bluefs-volume-ops 2026-02-17T21:38:09.411 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:406: TEST_bluestore_expand: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:38:09.411 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:408: TEST_bluestore_expand: ulimit -n 2026-02-17T21:38:09.412 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:408: TEST_bluestore_expand: local flimit=4096 2026-02-17T21:38:09.412 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:409: TEST_bluestore_expand: '[' 4096 -lt 1536 ']' 2026-02-17T21:38:09.412 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:412: TEST_bluestore_expand: export CEPH_MON=127.0.0.1:7146 2026-02-17T21:38:09.412 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:412: TEST_bluestore_expand: CEPH_MON=127.0.0.1:7146 2026-02-17T21:38:09.412 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:413: TEST_bluestore_expand: export CEPH_ARGS 2026-02-17T21:38:09.412 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:414: TEST_bluestore_expand: uuidgen 2026-02-17T21:38:09.414 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:414: TEST_bluestore_expand: CEPH_ARGS+='--fsid=f42ac715-e4a1-4de1-95c6-9310a558f0fe --auth-supported=none ' 2026-02-17T21:38:09.414 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:415: TEST_bluestore_expand: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-17T21:38:09.414 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:416: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_size=4294967296 ' 2026-02-17T21:38:09.414 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:417: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_db_create=true ' 2026-02-17T21:38:09.414 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:418: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_db_size=1073741824 ' 2026-02-17T21:38:09.414 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:419: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_wal_create=false ' 2026-02-17T21:38:09.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:420: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_fsck_on_mount=true ' 2026-02-17T21:38:09.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:421: TEST_bluestore_expand: CEPH_ARGS+='--osd_pool_default_size=1 ' 2026-02-17T21:38:09.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:422: TEST_bluestore_expand: CEPH_ARGS+='--osd_pool_default_min_size=1 ' 2026-02-17T21:38:09.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:423: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_debug_enforce_settings=ssd ' 2026-02-17T21:38:09.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:425: TEST_bluestore_expand: run_mon td/osd-bluefs-volume-ops a 2026-02-17T21:38:09.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:38:09.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T21:38:09.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T21:38:09.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T21:38:09.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-bluefs-volume-ops/a 2026-02-17T21:38:09.415 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bluefs-volume-ops/a --run-dir=td/osd-bluefs-volume-ops 2026-02-17T21:38:09.436 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T21:38:09.436 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:38:09.436 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:38:09.437 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:38:09.437 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:38:09.437 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:38:09.437 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:38:09.437 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bluefs-volume-ops/a '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bluefs-volume-ops/log --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T21:38:09.466 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T21:38:09.467 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T21:38:09.467 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:38:09.468 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:38:09.468 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T21:38:09.469 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T21:38:09.469 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:38:09.469 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:38:09.469 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:38:09.469 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:38:09.469 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:38:09.469 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:38:09.469 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86056/ceph-mon.a.asok 2026-02-17T21:38:09.470 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:38:09.470 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.86056/ceph-mon.a.asok config get fsid 2026-02-17T21:38:09.524 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T21:38:09.524 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T21:38:09.524 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T21:38:09.524 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T21:38:09.526 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T21:38:09.526 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T21:38:09.526 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T21:38:09.526 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T21:38:09.526 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T21:38:09.526 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:38:09.526 INFO:tasks.workunit.client.0.trial031.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:38:09.526 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86056/ceph-mon.a.asok 2026-02-17T21:38:09.527 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T21:38:09.527 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.86056/ceph-mon.a.asok config get mon_host 2026-02-17T21:38:09.580 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:426: TEST_bluestore_expand: run_mgr td/osd-bluefs-volume-ops x 2026-02-17T21:38:09.580 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:38:09.580 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T21:38:09.580 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T21:38:09.580 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T21:38:09.580 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-bluefs-volume-ops/x 2026-02-17T21:38:09.581 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T21:38:09.692 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T21:38:09.692 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:38:09.692 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:38:09.692 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:38:09.692 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:38:09.692 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:38:09.693 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:38:09.693 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:38:09.694 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bluefs-volume-ops/x '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T21:38:09.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:427: TEST_bluestore_expand: run_osd td/osd-bluefs-volume-ops 0 2026-02-17T21:38:09.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:38:09.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T21:38:09.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T21:38:09.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T21:38:09.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-17T21:38:09.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=9cb0c1bf-8dae-4aee-9568-2890d09706c5 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=f42ac715-e4a1-4de1-95c6-9310a558f0fe --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-17T21:38:09.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:38:09.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:38:09.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:38:09.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-17T21:38:09.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-17T21:38:09.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T21:38:09.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T21:38:09.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:38:09.718 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T21:38:09.718 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:38:09.718 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:38:09.719 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:38:09.719 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:38:09.719 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:38:09.719 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:38:09.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:38:09.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:38:09.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T21:38:09.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T21:38:09.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:38:09.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:38:09.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:38:09.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:38:09.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:38:09.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:38:09.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T21:38:09.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T21:38:09.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-17T21:38:09.720 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T21:38:09.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=0c841f6a-0298-4d22-b61e-b92fa6f8be94 2026-02-17T21:38:09.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 0c841f6a-0298-4d22-b61e-b92fa6f8be94' 2026-02-17T21:38:09.721 INFO:tasks.workunit.client.0.trial031.stdout:add osd0 0c841f6a-0298-4d22-b61e-b92fa6f8be94 2026-02-17T21:38:09.721 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T21:38:09.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDB35RpNcDWKxAAyzInCCa66TkJC2EnNkZ33g== 2026-02-17T21:38:09.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDB35RpNcDWKxAAyzInCCa66TkJC2EnNkZ33g=="}' 2026-02-17T21:38:09.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 0c841f6a-0298-4d22-b61e-b92fa6f8be94 -i td/osd-bluefs-volume-ops/0/new.json 2026-02-17T21:38:09.849 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:38:09.857 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/0/new.json 2026-02-17T21:38:09.858 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=9cb0c1bf-8dae-4aee-9568-2890d09706c5 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=f42ac715-e4a1-4de1-95c6-9310a558f0fe --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDB35RpNcDWKxAAyzInCCa66TkJC2EnNkZ33g== --osd-uuid 0c841f6a-0298-4d22-b61e-b92fa6f8be94 2026-02-17T21:38:09.877 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:38:09.874+0000 7f293f52a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:38:09.878 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:38:09.878+0000 7f293f52a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:38:09.879 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:38:09.878+0000 7f293f52a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:38:09.880 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:38:09.878+0000 7f293f52a8c0 -1 bdev(0x55e0da108c00 td/osd-bluefs-volume-ops/0/block) open stat got: (1) Operation not permitted 2026-02-17T21:38:09.880 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:38:09.878+0000 7f293f52a8c0 -1 bluestore(td/osd-bluefs-volume-ops/0) _read_fsid unparsable uuid 2026-02-17T21:38:15.234 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/0/keyring 2026-02-17T21:38:15.234 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T21:38:15.235 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T21:38:15.235 INFO:tasks.workunit.client.0.trial031.stdout:adding osd0 key to auth repository 2026-02-17T21:38:15.236 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T21:38:15.511 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:38:15.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T21:38:15.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=9cb0c1bf-8dae-4aee-9568-2890d09706c5 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=f42ac715-e4a1-4de1-95c6-9310a558f0fe --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:38:15.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T21:38:15.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T21:38:15.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T21:38:15.540 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:38:15.538+0000 7fb3cbac48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:38:15.542 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:38:15.542+0000 7fb3cbac48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:38:15.543 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:38:15.542+0000 7fb3cbac48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:38:15.717 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T21:38:15.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:38:15.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:38:15.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:38:15.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:38:15.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:38:15.718 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:38:15.718 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:38:15.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:38:15.719 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:38:15.917 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:38:16.919 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:38:16.919 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:38:16.920 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:38:16.920 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:38:16.920 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:38:16.921 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:38:17.121 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:38:17.311 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:38:17.310+0000 7fb3cbac48c0 -1 Falling back to public interface 2026-02-17T21:38:18.123 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:38:18.123 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:38:18.123 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:38:18.124 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:38:18.124 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:38:18.124 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:38:18.323 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:38:19.325 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:38:19.325 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:38:19.326 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:38:19.326 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:38:19.326 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:38:19.326 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:38:19.522 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:38:20.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:38:20.524 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:38:20.525 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:38:20.525 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:38:20.525 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:38:20.525 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:38:20.744 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:38:21.746 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:38:21.746 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:38:21.747 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:38:21.747 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:38:21.747 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:38:21.747 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:38:21.949 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:38:22.159 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:38:22.158+0000 7fb3cbac48c0 -1 osd.0 0 log_to_monitors true 2026-02-17T21:38:22.950 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:38:22.950 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:38:22.951 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:38:22.951 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:38:22.951 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:38:22.952 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:38:23.150 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3530094212,v1:127.0.0.1:6803/3530094212] [v2:127.0.0.1:6804/3530094212,v1:127.0.0.1:6805/3530094212] exists,up 0c841f6a-0298-4d22-b61e-b92fa6f8be94 2026-02-17T21:38:23.150 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:38:23.150 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:38:23.150 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:38:23.151 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:428: TEST_bluestore_expand: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-17T21:38:23.152 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:428: TEST_bluestore_expand: osd_pid0=141709 2026-02-17T21:38:23.152 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:430: TEST_bluestore_expand: sleep 5 2026-02-17T21:38:28.153 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:431: TEST_bluestore_expand: create_pool foo 16 2026-02-17T21:38:28.154 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 16 2026-02-17T21:38:28.411 INFO:tasks.workunit.client.0.trial031.stderr:pool 'foo' already exists 2026-02-17T21:38:28.421 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T21:38:29.422 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:434: TEST_bluestore_expand: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-17T21:38:29.446 INFO:tasks.workunit.client.0.trial031.stdout:hints = 1 2026-02-17T21:38:29.446 INFO:tasks.workunit.client.0.trial031.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-17T21:38:29.446 INFO:tasks.workunit.client.0.trial031.stdout:Object prefix: benchmark_data_trial031_143794 2026-02-17T21:38:29.446 INFO:tasks.workunit.client.0.trial031.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-17T21:38:29.446 INFO:tasks.workunit.client.0.trial031.stdout: 0 0 0 0 0 0 - 0 2026-02-17T21:38:30.446 INFO:tasks.workunit.client.0.trial031.stdout: 1 16 14010 13994 54.6597 54.6641 0.000920733 0.00114182 2026-02-17T21:38:31.446 INFO:tasks.workunit.client.0.trial031.stdout: 2 16 28132 28116 54.9098 55.1641 0.00126556 0.00113696 2026-02-17T21:38:32.446 INFO:tasks.workunit.client.0.trial031.stdout: 3 16 42335 42319 55.0987 55.4805 0.00118808 0.00113319 2026-02-17T21:38:33.446 INFO:tasks.workunit.client.0.trial031.stdout: 4 16 56441 56425 55.0984 55.1016 0.00092413 0.0011332 2026-02-17T21:38:34.446 INFO:tasks.workunit.client.0.trial031.stdout: 5 16 70491 70475 55.0545 54.8828 0.000918679 0.00113417 2026-02-17T21:38:35.446 INFO:tasks.workunit.client.0.trial031.stdout: 6 16 84618 84602 55.0753 55.1836 0.00109491 0.00113374 2026-02-17T21:38:36.446 INFO:tasks.workunit.client.0.trial031.stdout: 7 16 98600 98584 55.0093 54.6172 0.00140492 0.00113512 2026-02-17T21:38:37.447 INFO:tasks.workunit.client.0.trial031.stdout: 8 16 111741 111725 54.5491 51.332 0.00116584 0.0011447 2026-02-17T21:38:38.447 INFO:tasks.workunit.client.0.trial031.stdout: 9 16 124224 124208 53.9057 48.7617 0.00141959 0.0011584 2026-02-17T21:38:39.447 INFO:tasks.workunit.client.0.trial031.stdout: 10 16 136676 136660 53.3788 48.6406 0.00178563 0.00116983 2026-02-17T21:38:40.447 INFO:tasks.workunit.client.0.trial031.stdout: 11 16 149317 149301 53.0148 49.3789 0.00101469 0.00117785 2026-02-17T21:38:41.447 INFO:tasks.workunit.client.0.trial031.stdout: 12 16 162718 162702 52.9589 52.3477 0.000710264 0.00117915 2026-02-17T21:38:42.447 INFO:tasks.workunit.client.0.trial031.stdout: 13 16 175827 175811 52.8238 51.207 0.00170432 0.00118218 2026-02-17T21:38:43.447 INFO:tasks.workunit.client.0.trial031.stdout: 14 15 188218 188203 52.508 48.4062 0.00210298 0.00118928 2026-02-17T21:38:44.447 INFO:tasks.workunit.client.0.trial031.stdout: 15 15 200352 200337 52.1671 47.3984 0.00125643 0.00119706 2026-02-17T21:38:45.447 INFO:tasks.workunit.client.0.trial031.stdout: 16 16 212429 212413 51.8547 47.1719 0.00108649 0.00120431 2026-02-17T21:38:46.447 INFO:tasks.workunit.client.0.trial031.stdout: 17 16 224454 224438 51.5673 46.9727 0.00108388 0.00121102 2026-02-17T21:38:47.447 INFO:tasks.workunit.client.0.trial031.stdout: 18 16 236478 236462 51.3116 46.9688 0.00108509 0.00121707 2026-02-17T21:38:48.447 INFO:tasks.workunit.client.0.trial031.stdout: 19 16 248456 248440 51.0734 46.7891 0.00133322 0.00122272 2026-02-17T21:38:49.447 INFO:tasks.workunit.client.0.trial031.stdout:2026-02-17T21:38:49.450211+0000 min lat: 0.000525563 max lat: 0.00790424 avg lat: 0.0012282 2026-02-17T21:38:49.448 INFO:tasks.workunit.client.0.trial031.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-17T21:38:49.448 INFO:tasks.workunit.client.0.trial031.stdout: 20 16 260370 260354 50.8465 46.5391 0.00122018 0.0012282 2026-02-17T21:38:50.448 INFO:tasks.workunit.client.0.trial031.stdout: 21 16 272385 272369 50.66 46.9336 0.00103903 0.00123273 2026-02-17T21:38:51.448 INFO:tasks.workunit.client.0.trial031.stdout: 22 16 284505 284489 50.5091 47.3438 0.00386194 0.00123639 2026-02-17T21:38:52.448 INFO:tasks.workunit.client.0.trial031.stdout: 23 16 296525 296509 50.3543 46.9531 0.00151071 0.00124023 2026-02-17T21:38:53.448 INFO:tasks.workunit.client.0.trial031.stdout: 24 16 308306 308290 50.1735 46.0195 0.00115527 0.0012447 2026-02-17T21:38:54.448 INFO:tasks.workunit.client.0.trial031.stdout: 25 16 319932 319916 49.983 45.4141 0.0019614 0.00124944 2026-02-17T21:38:55.448 INFO:tasks.workunit.client.0.trial031.stdout: 26 16 331549 331533 49.8058 45.3789 0.00111799 0.00125388 2026-02-17T21:38:56.448 INFO:tasks.workunit.client.0.trial031.stdout: 27 16 343085 343069 49.63 45.0625 0.00124836 0.00125834 2026-02-17T21:38:57.448 INFO:tasks.workunit.client.0.trial031.stdout: 28 16 354959 354943 49.5139 46.3828 0.00136038 0.00126129 2026-02-17T21:38:58.448 INFO:tasks.workunit.client.0.trial031.stdout: 29 16 366715 366699 49.3899 45.9219 0.00147308 0.00126445 2026-02-17T21:38:59.448 INFO:tasks.workunit.client.0.trial031.stdout: 30 16 379297 379281 49.3818 49.1484 0.00188508 0.00126272 2026-02-17T21:39:00.449 INFO:tasks.workunit.client.0.trial031.stdout:Total time run: 30.3217 2026-02-17T21:39:00.449 INFO:tasks.workunit.client.0.trial031.stdout:Total writes made: 379297 2026-02-17T21:39:00.449 INFO:tasks.workunit.client.0.trial031.stdout:Write size: 4096 2026-02-17T21:39:00.449 INFO:tasks.workunit.client.0.trial031.stdout:Object size: 4096 2026-02-17T21:39:00.449 INFO:tasks.workunit.client.0.trial031.stdout:Bandwidth (MB/sec): 48.8636 2026-02-17T21:39:00.449 INFO:tasks.workunit.client.0.trial031.stdout:Stddev Bandwidth: 3.59326 2026-02-17T21:39:00.449 INFO:tasks.workunit.client.0.trial031.stdout:Max bandwidth (MB/sec): 55.4805 2026-02-17T21:39:00.449 INFO:tasks.workunit.client.0.trial031.stdout:Min bandwidth (MB/sec): 45.0625 2026-02-17T21:39:00.450 INFO:tasks.workunit.client.0.trial031.stdout:Average IOPS: 12509 2026-02-17T21:39:00.450 INFO:tasks.workunit.client.0.trial031.stdout:Stddev IOPS: 919.875 2026-02-17T21:39:00.450 INFO:tasks.workunit.client.0.trial031.stdout:Max IOPS: 14203 2026-02-17T21:39:00.450 INFO:tasks.workunit.client.0.trial031.stdout:Min IOPS: 11536 2026-02-17T21:39:00.450 INFO:tasks.workunit.client.0.trial031.stdout:Average Latency(s): 0.00127588 2026-02-17T21:39:00.450 INFO:tasks.workunit.client.0.trial031.stdout:Stddev Latency(s): 0.00213747 2026-02-17T21:39:00.450 INFO:tasks.workunit.client.0.trial031.stdout:Max latency(s): 0.366658 2026-02-17T21:39:00.450 INFO:tasks.workunit.client.0.trial031.stdout:Min latency(s): 0.000525563 2026-02-17T21:39:00.531 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:435: TEST_bluestore_expand: sleep 5 2026-02-17T21:39:05.533 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:437: TEST_bluestore_expand: ceph tell osd.0 perf dump bluefs 2026-02-17T21:39:05.534 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:437: TEST_bluestore_expand: jq .bluefs.slow_total_bytes 2026-02-17T21:39:05.613 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:437: TEST_bluestore_expand: total_space_before=4294967296 2026-02-17T21:39:05.614 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: ceph tell osd.0 bluestore bluefs device info 2026-02-17T21:39:05.614 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: grep BDEV_SLOW -A 2 2026-02-17T21:39:05.614 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: grep free 2026-02-17T21:39:05.614 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: cut -d: -f 2 2026-02-17T21:39:05.614 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: cut -d, -f 1 2026-02-17T21:39:05.614 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: cut '-d ' -f 2 2026-02-17T21:39:05.692 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: free_space_before=2741276672 2026-02-17T21:39:05.692 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: kill 141709 2026-02-17T21:39:05.692 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: sleep 1 2026-02-17T21:39:06.693 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: kill 141709 2026-02-17T21:39:06.693 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: sleep 1 2026-02-17T21:39:07.694 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: kill 141709 2026-02-17T21:39:07.694 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 441: kill: (141709) - No such process 2026-02-17T21:39:07.695 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:442: TEST_bluestore_expand: ceph osd down 0 2026-02-17T21:39:07.897 INFO:tasks.workunit.client.0.trial031.stderr:osd.0 is already down. 2026-02-17T21:39:07.906 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:445: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 allocmap 2026-02-17T21:39:07.929 INFO:tasks.workunit.client.0.trial031.stdout:allocmap bluestore.allocmap 2026-02-17T21:39:50.056 INFO:tasks.workunit.client.0.trial031.stdout:allocmap success 2026-02-17T21:39:50.094 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:448: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 fsck 2026-02-17T21:40:33.214 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:40:33.264 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:450: TEST_bluestore_expand: requested_space=4294967296 2026-02-17T21:40:33.264 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:451: TEST_bluestore_expand: truncate td/osd-bluefs-volume-ops/0/block -s 4294967296 2026-02-17T21:40:33.265 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:452: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 bluefs-bdev-expand 2026-02-17T21:40:33.287 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:41:11.871 INFO:tasks.workunit.client.0.trial031.stdout:1 : device size 0x40000000(1 GiB) : using 0x19300000(403 MiB) 2026-02-17T21:41:11.871 INFO:tasks.workunit.client.0.trial031.stdout:2 : device size 0x100000000(4 GiB) : using 0x5c9b9000(1.4 GiB) 2026-02-17T21:41:11.872 INFO:tasks.workunit.client.0.trial031.stdout:Expanding DB/WAL... 2026-02-17T21:41:11.872 INFO:tasks.workunit.client.0.trial031.stdout:1 : nothing to do, skipped 2026-02-17T21:41:11.872 INFO:tasks.workunit.client.0.trial031.stdout:2 : nothing to do, skipped 2026-02-17T21:41:12.418 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:455: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 fsck 2026-02-17T21:41:56.783 INFO:tasks.workunit.client.0.trial031.stdout:fsck success 2026-02-17T21:41:56.834 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:458: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 qfsck 2026-02-17T21:41:56.850 INFO:tasks.workunit.client.0.trial031.stdout:qfsck bluestore.quick-fsck 2026-02-17T21:42:38.199 INFO:tasks.workunit.client.0.trial031.stdout:qfsck success 2026-02-17T21:42:38.243 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:460: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 bluefs-bdev-sizes 2026-02-17T21:42:38.260 INFO:tasks.workunit.client.0.trial031.stdout:inferring bluefs devices from bluestore path 2026-02-17T21:43:18.110 INFO:tasks.workunit.client.0.trial031.stdout:1 : device size 0x40000000(1 GiB) : using 0x19300000(403 MiB) 2026-02-17T21:43:18.110 INFO:tasks.workunit.client.0.trial031.stdout:2 : device size 0x100000000(4 GiB) : using 0x5c9b9000(1.4 GiB) 2026-02-17T21:43:18.885 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:462: TEST_bluestore_expand: activate_osd td/osd-bluefs-volume-ops 0 2026-02-17T21:43:18.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:43:18.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T21:43:18.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T21:43:18.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T21:43:18.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-17T21:43:18.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=9cb0c1bf-8dae-4aee-9568-2890d09706c5 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=f42ac715-e4a1-4de1-95c6-9310a558f0fe --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-17T21:43:18.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T21:43:18.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T21:43:18.886 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T21:43:18.887 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-17T21:43:18.887 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-17T21:43:18.887 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T21:43:18.887 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T21:43:18.887 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-17T21:43:18.887 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T21:43:18.887 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T21:43:18.887 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T21:43:18.887 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T21:43:18.887 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:43:18.887 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:43:18.887 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:43:18.887 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' 2026-02-17T21:43:18.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T21:43:18.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-17T21:43:18.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-17T21:43:18.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T21:43:18.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T21:43:18.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T21:43:18.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T21:43:18.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T21:43:18.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T21:43:18.888 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-17T21:43:18.889 INFO:tasks.workunit.client.0.trial031.stdout:start osd.0 2026-02-17T21:43:18.890 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T21:43:18.890 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=f79d0a89-b67f-4767-918b-e42caf0a6032 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=9cb0c1bf-8dae-4aee-9568-2890d09706c5 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=f42ac715-e4a1-4de1-95c6-9310a558f0fe --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.86056/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T21:43:18.890 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-17T21:43:18.890 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T21:43:18.890 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T21:43:18.890 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T21:43:18.890 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T21:43:18.911 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:43:18.911+0000 7fa0af3378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:43:18.912 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:43:18.911+0000 7fa0af3378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:43:18.914 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:43:18.911+0000 7fa0af3378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T21:43:19.089 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T21:43:19.090 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T21:43:19.090 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T21:43:19.090 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T21:43:19.090 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T21:43:19.090 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:43:19.091 INFO:tasks.workunit.client.0.trial031.stdout:0 2026-02-17T21:43:19.091 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T21:43:19.091 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:43:19.091 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:43:19.298 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:43:20.299 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:43:20.299 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:43:20.300 INFO:tasks.workunit.client.0.trial031.stdout:1 2026-02-17T21:43:20.300 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T21:43:20.300 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:43:20.301 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:43:20.420 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:43:20.419+0000 7fa0af3378c0 -1 Falling back to public interface 2026-02-17T21:43:20.510 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:43:21.511 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:43:21.511 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:43:21.512 INFO:tasks.workunit.client.0.trial031.stdout:2 2026-02-17T21:43:21.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T21:43:21.512 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:43:21.513 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:43:21.720 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:43:22.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:43:22.721 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:43:22.722 INFO:tasks.workunit.client.0.trial031.stdout:3 2026-02-17T21:43:22.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T21:43:22.722 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:43:22.723 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:43:22.926 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:43:23.927 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:43:23.928 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:43:23.928 INFO:tasks.workunit.client.0.trial031.stdout:4 2026-02-17T21:43:23.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T21:43:23.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:43:23.929 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:43:24.126 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:43:25.127 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:43:25.127 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:43:25.128 INFO:tasks.workunit.client.0.trial031.stdout:5 2026-02-17T21:43:25.128 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T21:43:25.128 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:43:25.128 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:43:25.330 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:43:26.331 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:43:26.331 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:43:26.331 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-17T21:43:26.332 INFO:tasks.workunit.client.0.trial031.stdout:6 2026-02-17T21:43:26.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:43:26.332 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:43:26.532 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:43:27.533 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:43:27.533 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:43:27.534 INFO:tasks.workunit.client.0.trial031.stdout:7 2026-02-17T21:43:27.534 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-17T21:43:27.535 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:43:27.535 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:43:27.733 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:43:28.734 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:43:28.735 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:43:28.735 INFO:tasks.workunit.client.0.trial031.stdout:8 2026-02-17T21:43:28.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-17T21:43:28.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:43:28.736 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:43:28.945 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:43:29.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:43:29.946 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:43:29.946 INFO:tasks.workunit.client.0.trial031.stdout:9 2026-02-17T21:43:29.947 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-17T21:43:29.947 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:43:29.947 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:43:30.145 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:43:31.146 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:43:31.147 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:43:31.147 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-17T21:43:31.147 INFO:tasks.workunit.client.0.trial031.stdout:10 2026-02-17T21:43:31.147 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:43:31.148 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:43:31.557 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:43:32.559 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:43:32.559 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:43:32.560 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 11 2026-02-17T21:43:32.560 INFO:tasks.workunit.client.0.trial031.stdout:11 2026-02-17T21:43:32.561 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:43:32.561 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:43:32.759 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:43:33.760 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:43:33.760 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:43:33.760 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 12 2026-02-17T21:43:33.761 INFO:tasks.workunit.client.0.trial031.stdout:12 2026-02-17T21:43:33.761 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:43:33.761 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:43:33.957 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T21:43:34.069 INFO:tasks.workunit.client.0.trial031.stderr:2026-02-17T21:43:34.067+0000 7fa0af3378c0 -1 osd.0 9 log_to_monitors true 2026-02-17T21:43:34.958 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T21:43:34.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T21:43:34.959 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 13 2026-02-17T21:43:34.959 INFO:tasks.workunit.client.0.trial031.stdout:13 2026-02-17T21:43:34.960 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T21:43:34.960 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T21:43:35.160 INFO:tasks.workunit.client.0.trial031.stdout:osd.0 up in weight 1 up_from 12 up_thru 12 down_at 9 last_clean_interval [5,9) [v2:127.0.0.1:6802/3176855975,v1:127.0.0.1:6803/3176855975] [v2:127.0.0.1:6804/3176855975,v1:127.0.0.1:6805/3176855975] exists,up 0c841f6a-0298-4d22-b61e-b92fa6f8be94 2026-02-17T21:43:35.161 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T21:43:35.161 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T21:43:35.161 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T21:43:35.161 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:463: TEST_bluestore_expand: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-17T21:43:35.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:463: TEST_bluestore_expand: osd_pid0=146167 2026-02-17T21:43:35.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:465: TEST_bluestore_expand: wait_for_clean 2026-02-17T21:43:35.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T21:43:35.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T21:43:35.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T21:43:35.162 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T21:43:35.163 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T21:43:35.163 INFO:tasks.workunit.client.0.trial031.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T21:43:35.163 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T21:43:35.163 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T21:43:35.163 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T21:43:35.258 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T21:43:35.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T21:43:35.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T21:43:35.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T21:43:35.259 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T21:43:35.259 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T21:43:35.467 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T21:43:35.467 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T21:43:35.467 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T21:43:35.468 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T21:43:35.548 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607555 2026-02-17T21:43:35.548 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607555 2026-02-17T21:43:35.548 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-51539607555' 2026-02-17T21:43:35.549 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T21:43:35.549 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-51539607555 2026-02-17T21:43:35.549 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T21:43:35.550 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T21:43:35.551 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-51539607555 2026-02-17T21:43:35.551 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T21:43:35.552 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607555 2026-02-17T21:43:35.552 INFO:tasks.workunit.client.0.trial031.stdout:waiting osd.0 seq 51539607555 2026-02-17T21:43:35.553 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 51539607555' 2026-02-17T21:43:35.553 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:43:35.764 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 51539607555 2026-02-17T21:43:35.765 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T21:43:36.766 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T21:43:36.766 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T21:43:36.969 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607555 -lt 51539607555 2026-02-17T21:43:36.969 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T21:43:36.970 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:43:36.970 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:43:37.230 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 == 0 2026-02-17T21:43:37.230 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T21:43:37.230 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T21:43:37.230 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T21:43:37.230 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T21:43:37.230 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T21:43:37.230 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T21:43:37.230 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T21:43:37.429 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=16 2026-02-17T21:43:37.429 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T21:43:37.430 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T21:43:37.430 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T21:43:37.690 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 16 = 16 2026-02-17T21:43:37.691 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T21:43:37.691 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T21:43:37.691 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:467: TEST_bluestore_expand: ceph tell osd.0 perf dump bluefs 2026-02-17T21:43:37.692 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:467: TEST_bluestore_expand: jq .bluefs.slow_total_bytes 2026-02-17T21:43:37.771 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:467: TEST_bluestore_expand: total_space_after=4294967296 2026-02-17T21:43:37.772 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: ceph tell osd.0 bluestore bluefs device info 2026-02-17T21:43:37.772 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: grep BDEV_SLOW -A 2 2026-02-17T21:43:37.772 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: grep free 2026-02-17T21:43:37.773 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: cut -d: -f 2 2026-02-17T21:43:37.773 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: cut -d, -f 1 2026-02-17T21:43:37.773 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: cut '-d ' -f 2 2026-02-17T21:43:37.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: free_space_after=2731515904 2026-02-17T21:43:37.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:470: TEST_bluestore_expand: '[' 4294967296 '!=' 4294967296 ']' 2026-02-17T21:43:37.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:476: TEST_bluestore_expand: total_space_added=0 2026-02-17T21:43:37.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:477: TEST_bluestore_expand: free_space_added=-9760768 2026-02-17T21:43:37.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:479: TEST_bluestore_expand: let 'new_used_space=(0 - -9760768)' 2026-02-17T21:43:37.852 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:480: TEST_bluestore_expand: echo 9760768 2026-02-17T21:43:37.852 INFO:tasks.workunit.client.0.trial031.stdout:9760768 2026-02-17T21:43:37.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:482: TEST_bluestore_expand: '[' 9760768 -gt 131072 ']' 2026-02-17T21:43:37.853 INFO:tasks.workunit.client.0.trial031.stdout:total_space_added = 0 2026-02-17T21:43:37.853 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:483: TEST_bluestore_expand: echo 'total_space_added = 0' 2026-02-17T21:43:37.854 INFO:tasks.workunit.client.0.trial031.stdout:free_space_added = -9760768 2026-02-17T21:43:37.855 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:484: TEST_bluestore_expand: echo 'free_space_added = -9760768' 2026-02-17T21:43:37.855 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:485: TEST_bluestore_expand: return 1 2026-02-17T21:43:37.855 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: return 1 2026-02-17T21:43:37.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2379: main: code=1 2026-02-17T21:43:37.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-bluefs-volume-ops 1 2026-02-17T21:43:37.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-17T21:43:37.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=1 2026-02-17T21:43:37.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-17T21:43:37.856 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T21:43:37.856 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T21:43:37.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T21:43:37.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T21:43:37.856 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T21:43:38.160 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T21:43:38.161 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T21:43:38.162 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T21:43:38.162 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T21:43:38.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T21:43:38.164 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T21:43:38.165 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T21:43:38.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:43:38.166 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T21:43:38.167 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T21:43:38.167 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T21:43:38.168 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:43:38.169 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 1 = 1 ']' 2026-02-17T21:43:38.169 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: '[' -n '' ']' 2026-02-17T21:43:38.169 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:194: teardown: mkdir -p /home/ubuntu/cephtest/archive/log 2026-02-17T21:43:38.170 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:195: teardown: mv td/osd-bluefs-volume-ops/bluestore_tool.log td/osd-bluefs-volume-ops/mgr.x.log td/osd-bluefs-volume-ops/mon.a.log td/osd-bluefs-volume-ops/osd.0.log /home/ubuntu/cephtest/archive/log 2026-02-17T21:43:38.171 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-17T21:43:38.238 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T21:43:38.239 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T21:43:38.239 INFO:tasks.workunit.client.0.trial031.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86056 2026-02-17T21:43:38.239 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86056 2026-02-17T21:43:38.240 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T21:43:38.240 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T21:43:38.240 INFO:tasks.workunit.client.0.trial031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 1 2026-02-17T21:43:38.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T21:43:38.241 INFO:tasks.workunit:Stopping ['osd'] on client.0... 2026-02-17T21:43:38.241 DEBUG:teuthology.orchestra.run.trial031:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-17T21:43:38.588 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_d855f53b89fdcec760fd9232a5fb55ed4fb111a1/qa/tasks/workunit.py", line 144, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_d855f53b89fdcec760fd9232a5fb55ed4fb111a1/qa/tasks/workunit.py", line 292, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_d855f53b89fdcec760fd9232a5fb55ed4fb111a1/qa/tasks/workunit.py", line 433, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/remote.py", line 575, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test osd/osd-bluefs-volume-ops.sh) on trial031 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d855f53b89fdcec760fd9232a5fb55ed4fb111a1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh' 2026-02-17T21:43:38.592 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-17T21:43:38.607 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-17T21:43:38.608 DEBUG:teuthology.orchestra.run.trial031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-17T21:43:38.714 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-17T21:43:38.726 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-17T21:43:38.726 DEBUG:teuthology.orchestra.run.trial031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T21:43:38.801 INFO:teuthology.orchestra.run.trial031.stdout: remote refid st t when poll reach delay offset jitter 2026-02-17T21:43:38.802 INFO:teuthology.orchestra.run.trial031.stdout:============================================================================== 2026-02-17T21:43:38.802 INFO:teuthology.orchestra.run.trial031.stdout:-5.78.90.46 (jmp 53.108.101.33 3 u 382 64 340 80.406 -4.602 2.106 2026-02-17T21:43:38.802 INFO:teuthology.orchestra.run.trial031.stdout:+triton.ellipse. 50.43.156.178 2 u 7 64 377 32.558 -1.977 0.961 2026-02-17T21:43:38.802 INFO:teuthology.orchestra.run.trial031.stdout:+172-235-154-118 40.119.6.228 4 u 61 64 377 36.899 -1.174 1.939 2026-02-17T21:43:38.802 INFO:teuthology.orchestra.run.trial031.stdout:*as393746.mci.tr 42.20.202.230 2 u 53 64 377 29.783 -1.403 1.347 2026-02-17T21:43:38.803 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-17T21:43:38.814 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-17T21:43:38.814 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-17T21:43:38.829 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-17T21:43:38.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-17T21:43:38.858 INFO:teuthology.task.internal:Duration was 1830.110805 seconds 2026-02-17T21:43:38.858 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-17T21:43:38.873 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-17T21:43:38.873 DEBUG:teuthology.orchestra.run.trial031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-17T21:43:38.899 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-17T21:43:38.899 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial031.front.sepia.ceph.com 2026-02-17T21:43:38.900 DEBUG:teuthology.orchestra.run.trial031:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-17T21:43:38.949 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-17T21:43:38.949 DEBUG:teuthology.orchestra.run.trial031:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T21:43:39.153 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-17T21:43:39.154 DEBUG:teuthology.orchestra.run.trial031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T21:43:39.163 INFO:teuthology.orchestra.run.trial031.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T21:43:39.164 INFO:teuthology.orchestra.run.trial031.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T21:43:39.164 INFO:teuthology.orchestra.run.trial031.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T21:43:39.164 INFO:teuthology.orchestra.run.trial031.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-17T21:43:39.164 INFO:teuthology.orchestra.run.trial031.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-17T21:43:39.215 INFO:teuthology.orchestra.run.trial031.stderr: 81.7% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-17T21:43:39.217 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-17T21:43:39.236 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-17T21:43:39.237 DEBUG:teuthology.orchestra.run.trial031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-17T21:43:39.271 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-17T21:43:39.290 DEBUG:teuthology.orchestra.run.trial031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:43:39.322 INFO:teuthology.orchestra.run.trial031.stdout:kernel.core_pattern = core 2026-02-17T21:43:39.331 DEBUG:teuthology.orchestra.run.trial031:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-17T21:43:39.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T21:43:39.378 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-17T21:43:39.396 INFO:teuthology.task.internal:Transferring archived files... 2026-02-17T21:43:39.398 DEBUG:teuthology.misc:Transferring archived files from trial031:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial/53907/remote/trial031 2026-02-17T21:43:39.399 DEBUG:teuthology.orchestra.run.trial031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-17T23:23:15.596 INFO:teuthology.task.internal:Removing archive directory... 2026-02-17T23:23:15.597 DEBUG:teuthology.orchestra.run.trial031:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-17T23:23:18.953 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-17T23:23:18.972 INFO:teuthology.task.internal:Not uploading archives. 2026-02-17T23:23:18.973 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-17T23:23:18.990 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-17T23:23:18.990 DEBUG:teuthology.orchestra.run.trial031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-17T23:23:18.995 INFO:teuthology.orchestra.run.trial031.stdout: 1180220 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 17 23:23 /home/ubuntu/cephtest 2026-02-17T23:23:18.996 INFO:teuthology.orchestra.run.trial031.stdout: 1190252 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 17 21:14 /home/ubuntu/cephtest/mnt.0 2026-02-17T23:23:18.996 INFO:teuthology.orchestra.run.trial031.stdout: 1190253 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 17 21:27 /home/ubuntu/cephtest/mnt.0/client.0 2026-02-17T23:23:18.996 INFO:teuthology.orchestra.run.trial031.stdout: 1204996 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 17 21:27 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T23:23:18.996 INFO:teuthology.orchestra.run.trial031.stdout: 1204997 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Feb 17 21:43 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td 2026-02-17T23:23:18.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T23:23:18.997 INFO:teuthology.orchestra.run.trial031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2026-02-17T23:23:18.998 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5f66ecfb34c0370410e78b3ee641753d19da653b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial031 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-17T23:23:19.001 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-17T23:23:19.018 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-17T23:23:19.036 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on trial031 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-17T23:23:19.043 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} duration: 1830.110805273056 failure_reason: 'Command failed (workunit test osd/osd-bluefs-volume-ops.sh) on trial031 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d855f53b89fdcec760fd9232a5fb55ed4fb111a1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh''' flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-17T23:23:19.043 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T23:23:19.177 INFO:teuthology.run:FAIL